Skip to main content

Advertisement

SpringerLink
Log in

Assessing Pre-Service Science Teachers’ Scientific Reasoning Competencies

  • Published:
Research in Science Education Aims and scope Submit manuscript

Abstract

Scientific reasoning competencies are highlighted in science education policy papers and standard documents in various countries around the world and pre-service science teachers are asked to develop them during teacher training as part of their professional competencies. In order to monitor the development of pre-service science teachers’ scientific reasoning competencies during their course of studies and to enable evidence-based improvements of teacher training, instruments for the assessment of scientific reasoning competencies are needed. However, studies propose that the validity of most instruments for assessing scientific reasoning competencies available so far can be questioned. This study presents an English translation of an already developed German multiple-choice instrument to assess pre-service science teachers’ scientific reasoning competencies. A sample of N = 105 Australian pre-service science teachers participated in this study by answering the translated instrument. Quantitative (differential item functioning, Mantel–Haenszel statistic) and qualitative (think-aloud protocols) analyses provide validity evidence for the translated instrument. Furthermore, the interpretation of the data as an indicator for the participating Australian pre-service science teachers’ scientific reasoning competencies suggests that there is a need for a more explicit emphasis on scientific reasoning in Australian science teacher training.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • ACARA [Australian Curriculum, Assessment, and Reporting Authority] (2013). General capabilities. January 2013 Edition. Retrieved from http://docs.acara.edu.au/resources/General_Capabilities_2011.pdf.

  • AERA, APA, & NCME [American Educational Research Association, American Psychological Association, & National Council on Measurement in Education]. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.

    Google Scholar 

  • AITSL [Australian Institute for Teaching and School Leadership]. (2011). Australian Professional Standards for Teachers. Carlton South: Education Council Retrieved from https://www.aitsl.edu.au/docs/default-source/general/australian-professional-standands-for-teachers-20171006.pdf.

    Google Scholar 

  • ASTA [Australian Science Teacher Association]. (2009). National professional standards for highly accomplished teachers of science: Final draft. Deakin: ASTA.

    Google Scholar 

  • Baumert, J., & Kunter, M. (2013). The COACTIV model of teachers’ professional competence. In M. Kunter, J. Baumert, W. Blum, U. Klusmann, S. Krauss, & M. Neubrand (Eds.), Cognitive activation in the mathematics classroom and professional competence of teachers (pp. 25–48). Boston: Springer US.

    Chapter  Google Scholar 

  • Bond, T., & Fox, C. (2001). Applying the Rasch model: Fundamental measurement in the human sciences. Mahwah: Erlbaum.

    Book  Google Scholar 

  • Brennan, R., & Prediger, D. (1981). Coefficient kappa: Some uses, misuses, and alternatives. Educational and Psychological Measurement, 41, 687–699.

    Article  Google Scholar 

  • Burnham, K., & Anderson, D. (2004). Multimodel inference: Understanding AIC and BIC in model selection. Sociological Methods & Research, 33, 261–304.

    Article  Google Scholar 

  • Capps, D., & Crawford, B. (2013). Inquiry-based professional development: What does it take to support teachers in learning about inquiry and nature of science? International Journal of Science Education, 35(12), 1947–1978. https://doi.org/10.1080/09500693.2012.760209.

    Article  Google Scholar 

  • Ding, L., Wei, X., & Mollohan, K. (2016). Does higher education improve student scientific reasoning skills? International Journal of Science and Mathematics Education, 14, 619–634. https://doi.org/10.1007/s10763-014-9597-y.

    Article  Google Scholar 

  • Driver, R., Leach, J., Millar, R., & Scott, P. (1996). Young people’s images of science. Buckingham: Open University Press.

    Google Scholar 

  • Educational Policies Commission. (1966). Education and the spirit of science. Washington, DC: National Education Association.

    Google Scholar 

  • Embretson, S., & Reise, S. (2000). Item response theory for psychologists. Mahwah, NJ: Erlbaum.

    Google Scholar 

  • Ercikan, K., & Lyons-Thomas, J. (2013). Adapting tests for use in other languages and cultures. In K. Geisinger (Ed.), APA handbook of testing and assessment in psychology. Testing and assessment in school psychology and education (pp. 545–569). Washington, DC: American Psychological Association.

    Chapter  Google Scholar 

  • Ercikan, K., Gierl, M., McCreith, T., Puhan, G., & Koh, K. (2004). Comparability of bilingual versions of assessments: Sources of incomparability of English and French versions of Canada’s National Achievement Tests. Applied Measurement in Education, 17, 301–321. https://doi.org/10.1207/s15324818ame1703_4.

    Article  Google Scholar 

  • Ercikan, K., Arim, R., Law, D., Domene, J., Gagnon, F., & Lacroix, S. (2010). Application of think aloud protocols for examining and confirming sources of differential item functioning identified by expert reviews. Educational Measurement: Issues and Practice, 29, 24–35. https://doi.org/10.1111/j.1745-3992.2010.00173.x.

    Article  Google Scholar 

  • Ericsson, K., & Simon, H. (1998). How to study thinking in everyday life: Contrasting think-aloud protocols with descriptions and explanations of thinking. Mind, Culture, and Activity, 5, 178–186.

    Article  Google Scholar 

  • European Commission. (2015). Science education for responsible citizenship. Brussels: European Commission Retrieved from http://ec.europa.eu/research/swafs/pdf/pub_science_education/KI-NA-26-893-EN-N.pdf.

    Google Scholar 

  • Forsyth, B., Kudela, M., Levin, K., Lawrence, D., & Willis, G. (2016). Methods for translating an English-language survey questionnaire on tobacco use into Mandarin, Cantonese, Korean, and Vietnamese. Field Methods, 19, 264–283. https://doi.org/10.1177/1525822X07302105.

    Article  Google Scholar 

  • Frey, A. (2006). Strukturierung und Methoden zur Erfassung von Kompetenz (Structuring and methods for competence assessment). Bildung und Erziehung, 59, 125–166.

    Article  Google Scholar 

  • Großschedl, J., Harms, U., Kleickmann, T., & Glowinski, I. (2015). Preservice biology teachers’ professional knowledge: Structure and learning opportunities. Journal of Science Teacher Education, 26(3), 291–318. https://doi.org/10.1007/s10972-015-9423-6.

    Article  Google Scholar 

  • Hanushek, E., & Woessmann, L. (2011). How much do educational outcomes matter in OECD countries? Economic Policy, 26, 427–491. https://doi.org/10.1111/j.1468-0327.2011.00265.x.

    Article  Google Scholar 

  • Harkness, J. (2003). Questionnaire translation. In J. Harkness, F. J. R. van de Vijver, & P. Mohler (Eds.), Cross-cultural survey methods (pp. 35–56). Hoboken: Wiley.

    Google Scholar 

  • Harkness, J., Pennell, B.-E., & Schoua-Glusberg, A. (2004). Survey questionnaire translation and assessment. In S. Presser, J. Rothgeb, M. Couper, J. Lessler, E. Martin, J. Martin, & E. Singer (Eds.), Methods for testing and evaluating survey questionnaires (pp. 453–473). Hoboken: Wiley.

    Chapter  Google Scholar 

  • Hartmann, S., Upmeier zu Belzen, A., Krüger, D., & Pant, H. (2015). Scientific reasoning in higher education. Zeitschrift für Psychologie, 223, 47–53. https://doi.org/10.1027/2151-2604/a000199.

    Article  Google Scholar 

  • Heijnes, D., van Joolingen, W., & Leenaars, F. (2017). Stimulating scientific reasoning with drawing-based modeling. Journal of Science Education and Technology, 333, 1096. https://doi.org/10.1007/s10956-017-9707-z.

    Article  Google Scholar 

  • Hodson, D. (2014). Learning science, learning about science, doing science: Different goals demand different learning methods. International Journal of Science Education, 36, 2534–2553. https://doi.org/10.1080/09500693.2014.899722.

    Article  Google Scholar 

  • Justi, R., & van Driel, J. (2005). A case study of the development of a beginning chemistry teacher's knowledge about models and modelling. Research in Science Education, 35, 197–219. https://doi.org/10.1007/s11165-004-7583-z.

    Article  Google Scholar 

  • Kane, M. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50, 1–73.

    Article  Google Scholar 

  • Kind, P., & Osborne, J. (2017). Styles of scientific reasoning: A cultural rationale for science education? Science Education, 101, 8–31. https://doi.org/10.1002/sce.21251.

    Article  Google Scholar 

  • Kleickmann, T., & Anders, Y. (2013). Learning at university. In M. Kunter, J. Baumert, W. Blum, U. Klusmann, S. Krauss, & M. Neubrand (Eds.), Cognitive activation in the mathematics classroom and professional competence of teachers (pp. 321–332). Boston: Springer US.

    Chapter  Google Scholar 

  • Klieme, E., Hartig, J., & Rauch, D. (2008). The concept of competence in educational contexts. In J. Hartig, E. Klieme, & D. Leutner (Eds.), Assessment of competencies in educational contexts (pp. 3–22). Göttingen: Hogrefe.

    Google Scholar 

  • KMK (Ed.). (2017). Ländergemeinsame inhaltliche Anforderungen für die Fachwissenschaften und Fachdidaktiken in der Lehrerbildung (Common guidelines for the subjects and the subject didactics in teacher education). Berlin. Retrieved from https://www.kmk.org/fileadmin/Dateien/veroeffentlichungen_beschluesse/2008/2008_10_16-Fachprofile-Lehrerbildung.pdf.

  • KMK [Sekretariat der Ständigen Konferenz der Kultusminister der Länder in der BRD]. (2005). Bildungsstandards im Fach Biologie für den Mittleren Schulabschluss (Biology education standards for the Mittlere Schulabschluss). München: Wolters Kluwer.

    Google Scholar 

  • Krell, M., & Krüger, D. (2015). Testing models: A key aspect to promote teaching activities related to models and modelling in biology lessons? Journal of Biological Education, 50, 160–173. https://doi.org/10.1080/00219266.2015.1028570.

    Article  Google Scholar 

  • Krell, M., Koska, J., Penning, F., & Krüger, D. (2015a). Fostering pre-service teachers’ views about nature of science: Evaluation of a new STEM curriculum. Research in Science & Technological Education, 33(3), 344–365. https://doi.org/10.1080/02635143.2015.1060411.

    Article  Google Scholar 

  • Krell, M., Reinisch, B., & Krüger, D. (2015b). Analyzing students’ understanding of models and modeling referring to the disciplines biology, chemistry, and physics. Research in Science Education, 45, 367–393. https://doi.org/10.1007/s11165-014-9427-9

  • Krell, M. (2017). Schwierigkeitserzeugende Aufgabenmerkmale bei Multiple-Choice-Aufgaben zur Experimentierkompetenz im Biologieunterricht: Eine Replikationsstudie [Difficulty generating task characteristics of multiple-choice-tasks to assess experimental competencies]. Zeitschrift für Didaktik der Naturwissenschaften. https://doi.org/10.1007/s40573-017-0069-0.

  • Krell, M., Walzer, C., Hergert, S., & Krüger, D. (2017). Development and Application of a Category System to Describe Pre-Service Science Teachers’ Activities in the Process of Scientific Modelling. Research in Science Education, 333, 1096. https://doi.org/10.1007/s11165-017-9657-8.

    Article  Google Scholar 

  • Krell, M., Vergara, C., van Driel, J., Upmeier zu Belzen, A., & Krüger, D. (2018). Assessing pre-service teachers' scientific reasoning competencies: translation of a German mc instrument into Spanish/ English. Paper presented at NARST conference 2018. USA: Atlanta, GA.

  • Kunter, M., Klusmann, U., Baumert, J., Richter, D., Voss, T., & Hachfeld, A. (2013). Professional competence of teachers: Effects on instructional quality and student development. Journal of Educational Psychology, 105, 805–820. https://doi.org/10.1037/a0032583.

    Article  Google Scholar 

  • Lawson, A. (2004). The nature and development of scientific reasoning: A synthetic view. International Journal of Science and Mathematics Education, 2, 307–338. https://doi.org/10.1007/s10763-004-3224-2.

    Article  Google Scholar 

  • Mathesius, S., Upmeier zu Belzen, A., & Krüger, D. (2014). Kompetenzen von Biologiestudierenden im Bereich der naturwissenschaftlichen Erkenntnisgewinnung: Entwicklung eines Testinstruments [Competencies of biology students in the field of scientific inquiry: Development of a testing instrument]. Erkenntnisweg Biologiedidaktik, 13, 73–88.

    Google Scholar 

  • Mathesius, S., Hartmann, S., Upmeier zu Belzen, A., & Krüger, D. (2016). Scientific reasoning as an aspect of pre-service biology teacher education. In T. Tal & A. Yarden (Eds.), The future of biology education research. Proceedings of the 10th conference of European Researchers in Didactics of Biology (ERIDOB) (pp. 93–110). Haifa, Israel.

  • Mathesius, S., Upmeier zu Belzen, A. & Krüger, D. (2018a). Eyetracking als Methode zur Untersuchung von Lösungsprozessen bei Multiple-Choice-Aufgaben zum wissenschaftlichen Denken. In: M. Hammann & M. Lindner (Hrsg.), Lehr- und Lernforschung in der Biologiedidaktik, Band 8 (pp. 225–244). Innsbruck: Studienverlag.

  • Mathesius, S., Upmeier zu Belzen, A. & Krüger, D. (2018b). Lautes Denken bei der Bearbeitung von Multiple Choice Aufgaben zur Erfassung von Kompetenzen des wissenschaftlichen Denkens (working title). Manuscript in preparation.

  • Mayer, J. (2007). Erkenntnisgewinnung als wissenschaftliches Problemlösen (Scientific inquiry as problem solving). In D. Krüger & H. Vogt (Eds.), Theorien in der biologiedidaktischen Forschung (pp. 177–186). Berlin: Springer.

  • Mayer, D., Sodian, B., Koerber, S., & Schwippert, K. (2014). Scientific reasoning in elementary school children: Assessment and relations with cognitive abilities. Learning and Instruction, 29, 43–55. https://doi.org/10.1016/j.learninstruc.2013.07.005.

    Article  Google Scholar 

  • Morris, B., Croker, S., Masnick, A., & Zimmerman, C. (2012). The emergence of scientific reasoning. In H. Kloos, B. Morris, & J. Amaral (Eds.), Current topics in children's learning and cognition (pp. 61–82). InTech.

  • Neumann, K., Härtig, H., Harms, U., & Parchmann, I. (2017). Science teacher preparation in Germany. In J. Pedersen, T. Isozaki, & T. Hirano (Eds.), Model science teacher preparation programs. An international comparison of what works (pp. 29–52). Information Age: Charlotte.

    Google Scholar 

  • NGSS Lead States (Ed.). (2013). Next generation science standards: For states, by states. Washington, DC: The National Academies Press.

    Google Scholar 

  • OECD. (2010). The high cost of low educational performance: The long-run economic impact of improving PISA outcomes. Paris. Retrieved from https://www.oecd.org/pisa/44417824.pdf.

  • Opitz, A., Heene, M., & Fischer, F. (2017). Measuring scientific reasoning: A review of test instruments. Educational Research and Evaluation, 23, 78–101. https://doi.org/10.1080/13803611.2017.1338586.

    Article  Google Scholar 

  • Osborne, J. (2013). The 21st century challenge for science education: Assessing scientific reasoning. Thinking Skills and Creativity, 10, 265–279. https://doi.org/10.1016/j.tsc.2013.07.006.

    Article  Google Scholar 

  • Osborne, J. (2014). Scientific practices and inquiry in the science classroom. In N. Lederman & S. Abell (Eds.), Handbook of research on science education (pp. 579–599). New York: Routledge.

    Google Scholar 

  • Pedersen, J. E., Isozaki, T., & Hirano, T. (Eds.). (2017). Model science teacher preparation programs: An international comparison of what works. Charlotte: Information Age.

    Google Scholar 

  • Roth, W.-M., Oliveri, M., Sandilands, D., Lyons-Thomas, J., & Ercikan, K. (2013). Investigating linguistic sources of differential item functioning using expert think-aloud protocols in science achievement tests. International Journal of Science Education, 35, 546–576. https://doi.org/10.1080/09500693.2012.721572.

    Article  Google Scholar 

  • Schauble, L., Klopfer, L., & Raghavan, K. (1991). Students’ transition from an engineering model to a science model of experimentation. Journal of Research in Science Teaching, 28, 859–882.

    Article  Google Scholar 

  • Shavelson, R. (2013). On an approach to testing and modeling competence. Educational Psychologist, 48, 73–86. https://doi.org/10.1080/00461520.2013.779483.

    Article  Google Scholar 

  • Schreier, M. (2012). Qualitative content analysis in practice. Thousand Oaks: Sage.

    Google Scholar 

  • Schwarz, C., & White, B. (2005). Metamodeling knowledge: Developing students’ understanding of scientific modeling. Cognition and Instruction, 23, 165–205.

    Article  Google Scholar 

  • Shulman, L. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15, 4–14.

    Article  Google Scholar 

  • Stiller, J., Hartmann, S., Mathesius, S., Straube, P., Tiemann, R., Nordmeier, V., … Upmeier zu Belzen, A. (2016). Assessing scientific reasoning: A comprehensive evaluation of item features that affect item difficulty. Assessment & Evaluation in Higher Education, 41, 721–732. doi:https://doi.org/10.1080/02602938.2016.1164830

  • Thompson, E., Bowling, B., & Markle, R. (2017). Predicting student success in a major’s introductory biology course via logistic regression analysis of scientific reasoning ability and mathematics scores. Research in Science Education, 30(2), 663–163. https://doi.org/10.1007/s11165-016-9563-5.

    Article  Google Scholar 

  • Upmeier zu Belzen, A., & Krüger, D. (2010). Modellkompetenz im Biologieunterricht [Model competence in biology teaching]. Zeitschrift für Didaktik der Naturwissenschaften, 16, 41–57.

    Google Scholar 

  • van der Graaf, J., Segers, E., & Verhoeven, L. (2016). Scientific reasoning in kindergarten: Cognitive factors in experimentation and evidence evaluation. Learning and Individual Differences, 49, 190–200. https://doi.org/10.1016/j.lindif.2016.06.006.

    Article  Google Scholar 

  • VCAA [Victorian Curriculum and Assessment Authority]. (2016a). Victorian certificate of education biology: Advice for teachers. Melbourne: VCAA.

    Google Scholar 

  • VCAA [Victorian Curriculum and Assessment Authority]. (2016b). Victorian Curriculum: F-10. Melbourne, VIC. Retrieved from http://victoriancurriculum.vcaa.vic.edu.au/science/curriculum/f-10.

  • Weinert, F. (2001). Concept of competence: A conceptual clarification. In D. Rychen & L. Salganik (Eds.), Defining and selecting key competencies (pp. 45–65). Kirkland: Hogrefe.

    Google Scholar 

  • White, B., Collins, A., & Frederiksen, J. (2011). The nature of scientific meta-knowledge. In M. Khine & I. Saleh (Eds.), Models and modeling. Cognitive tools for scientific enquiry (pp. 41–76). Dordrecht: Springer.

    Google Scholar 

  • Windschitl, M., Thompson, J., & Braaten, M. (2008). Beyond the scientific method: Model-based inquiry as a new paradigm of preference for school science investigations. Science Education, 92(5), 941–967. https://doi.org/10.1002/sce.20259.

    Article  Google Scholar 

  • Won, M., Hackling, M., & Treagust, D. (2017). Secondary science teacher education in Australia. In J. Pedersen, T. Isozaki, & T. Hirano (Eds.), Model science teacher preparation programs. An international comparison of what works (pp. 229–248). Information Age: Charlotte.

    Google Scholar 

  • Wu, M. L., Adams, R., Wilson, M., & Haldane, S. (2007). ACER ConQuest. Camberwell: ACER Press.

    Google Scholar 

  • Zwick, R., Thayer, D., & Lewis, C. (1999). An empirical Bayes approach to Mantel-Haenszel DIF analysis. Journal of Educational Measurement, 36, 1–28.

    Article  Google Scholar 

Download references

Acknowledgments

The authors thank the German Federal Ministry of Education and Research for funding the projects Ko-WADiS/ValiDiS (grant numbers 01PK11004A/01PK15004A) and the Center for International Cooperation of Freie Universität Berlin for funding the research in Melbourne.

Author information

Authors and Affiliations

  1. Freie Universität Berlin, Berlin, Germany

    Moritz Krell, Sabrina Mathesius & Dirk Krüger

  2. Melbourne Graduate School of Education, The University of Melbourne, Carlton, Australia

    Christine Redman & Jan van Driel

Authors
  1. Moritz Krell
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christine Redman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sabrina Mathesius
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dirk Krüger
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jan van Driel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Moritz Krell.

Appendix

Appendix

Table 8 Category system used for data analysis in this study (based on Mathesius et al. 2018b)
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Krell, M., Redman, C., Mathesius, S. et al. Assessing Pre-Service Science Teachers’ Scientific Reasoning Competencies. Res Sci Educ 50, 2305–2329 (2020). https://doi.org/10.1007/s11165-018-9780-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11165-018-9780-1

Keywords

Search

Navigation