Skip to main content
SpringerLink
Log in

DEVELOPMENT, EVALUATION, AND VALIDATION OF A PAPER-AND-PENCIL TEST FOR MEASURING TWO COMPONENTS OF BIOLOGY TEACHERS’ PEDAGOGICAL CONTENT KNOWLEDGE CONCERNING THE “CARDIOVASCULAR SYSTEM”

  • Published:
International Journal of Science and Mathematics Education Aims and scope Submit manuscript

ABSTRACT

One main focus of teacher education research concentrates on teachers’ pedagogical content knowledge (PCK). It has been shown that teachers’ PCK correlates with teaching effectiveness as well as with students’ achievement gains. Teachers’ PCK should be analyzed as one of the main important components to evaluate professional development programs. On this account, it is necessary to develop standardized measures of biology teachers’ topic-specific PCK that are labor-efficient. This paper presents a study on the development, evaluation, and validation of a paper-and-pencil test to measure biology teachers’ declarative PCK on the topic of blood and the human cardiovascular system. The development of the test was based, among other considerations, on a review of research literature on PCK and an analysis of 50 videotaped biology lessons. The final test instrument was comprised of 15 items distributed across 2 scales. The findings of the main study—with 93 preservice and in-service biology teachers and 12 biologists—confirmed that this measure of biology teachers’ declarative PCK was reliable, objective, and valid. In-service biology teachers scored higher on the test than preservice teachers (effect size Cohen’s d, 0.65) on one hand and, also, than biologists (Cohen’s d, 1.00) on the other hand. Future versions of this test should explore enlarging the scales and measuring procedural aspects of PCK.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Abell, S. K. (2008). Twenty years later: Does pedagogical content knowledge remain a useful idea? International Journal of Science Education, 30, 1405–1416.

    Article  Google Scholar 

  • Allen, M. J. & Yen, W. M. (2002). Introduction to measurement theory. Long Grove: Waveland.

    Google Scholar 

  • Anderson, J. R. (1981). Cognitive skills and their acquisition. Hillsdale: Lawrence Erlbaum.

    Google Scholar 

  • Schmelzing, S. (2010). Das fachdidaktische Wissen von Biologielehrkräften: Konzeptionalisierung, Diagnostik, Struktur und Entwicklung im Rahmen der Biologielehrerbildung. [Pedagogical content knowledge of biology teachers: conceptualization, diagnostics, structure and development within biology teacher education]. Berlin: Logos.

  • van Driel, J. H., Verloop, N. & de Vos, W. (1998). Developing science teachers' pedagogical content knowledge. Journal of Research in Science Teaching, 35, (6), 673–695.

    Google Scholar 

  • Schmelzing, S., Wüsten, S., Sandmann, A. & Neuhaus, B. (2008). Evaluation von zentralen Inhalten der Lehrerbildung: Ansätze zur Diagnostik des fachdidaktischen Wissens von Biologielehrkräften. [Evaluation of central aspects of teacher education: Initial stages of diagnosing the pedagogical content knowledge of biology teachers.] Lehrerbildung auf dem Prüfstand, 1, 641–663.

  • Schmelzing, S., Fuchs, Ch., Wüsten, S., Sandmann, A. & Neuhaus, B. (2009a). Entwicklung und Evaluation eines Instruments zur Erfassung des fachdidaktischen Reflexionswissens von Biologielehrkräften. [Development and Evaluation of an instrument to measure the reflection abilities of biology teachers with regard to pedagogical content knowledge.] Lehrerbildung auf dem Prüfstand, 2, 56–80.

  • Schmelzing, S., Wüsten, S, Sandmann, A. & Neuhaus, B. (2010a). Measuring declarative and reflective components of biology teachers‘ pedagogical content knowledge. In: M.F. Taşar & G. Çakmakcı (Eds.). Contemporary science education research: preservice and inservice teacher education. Ankara, Turkey: Pegem Akademi, pp. 71–77.

  • Schmelzing, S., Wüsten, S., Sandmann, A. & Neuhaus, B. (2010b). Fachdidaktisches Wissen und Reflektieren im Querschnitt der Biologielehrerbildung. [Pedagogical content knowledge and reflection in frame of biology teacher education.] Zeitschrift für Didaktik der Naturwissenschaften, 16, 189–207.

  • Jüttner, M. & Neuhaus, B. J. (2012). Development of items for a pedagogical content knowledge-test based on empirical analysis of students’ errors. International Journal of Science Education, 34, (7), 1125–1143.

  • Baumert, J., Blum, W. & Neubrand, M. (2004). Drawing the lessons from PISA 2000. Zeitschrift für Erziehungswissenschaft, Beiheft, 3, 143–157.

    Google Scholar 

  • Baumert, J., Kunter, M., Blum, W., Brunner, M., Voss, T., Jordan, A., Klusmann, U., Tsai, Y. M. et al. (2010). Teachers’ mathematical knowledge, cognitive activation in the classroom, and student progress. American Educational Research Journal, 47, 133–180.

    Google Scholar 

  • Baxter, J. A. & Lederman, N. G. (1999). Assessment and measurement of pedagogical content knowledge. In J. Gess-Newsome & N. G. Lederman (Eds.), Examining pedagogical content knowledge (pp. 147–161). Dordrecht: Kluwer Academic.

    Google Scholar 

  • Beaton, A. E., Martin, M. O., Mullis, I. V. S., Gonzalez, E. J., Smith, T. A. & Kelly, D. L. (1996). Science achievement in the middle school years: IEA’s Third International Mathematics and Science Study (TIMSS). Chestnut Hill: TIMMS International Study Center, Boston College.

    Google Scholar 

  • Carlson, R. E. (1990). Assessing teachers’ pedagogical content knowledge: Item development issues. Journal of Personal Evaluation in Education, 4, 157–163.

    Article  Google Scholar 

  • Cochran, K. F., King, R. A. & De Ruiter, J. A. (1993). Pedagogical content knowing: An integrative model for teacher preparation. Journal of Teacher Education, 44, 263–272.

    Article  Google Scholar 

  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences. New York: Lawrence Erlbaum.

    Google Scholar 

  • Fenstermacher, G. D. (1994). The knower and the known: The nature of knowledge in research on teaching. In L. Darling-Hammond (Ed.), Review of research in education (20th ed., pp. 3–56). Washington, DC: American Educational Research Association.

    Google Scholar 

  • Fernandez-Balboa, J.-M. & Stiehl, J. (1995). The generic nature of pedagogical content knowledge among college professors. Teaching and Teacher Education, 11, 293–306.

    Article  Google Scholar 

  • Gardner, A. L., & Gess-Newsome, J. (2011, April). A rubric to measure teachers’ knowledge of inquiry-based instruction using three data sources. Paper presented at the NARST Annual Meeting, Orlando.

  • Geddis, A. N. (1993). Transforming subject-matter knowledge: The role of pedagogical content knowledge in learning to reflect on teaching. International Journal of Science Education, 15, 673–683.

    Article  Google Scholar 

  • Gess-Newsome, J. (1999). Pedagogical content knowledge: An introduction and orientation. In J. Gess-Newsome & N. G. Lederman (Eds.), Examining pedagogical content knowledge (pp. 3–17). Dordrecht: Kluwer Academic.

    Google Scholar 

  • Gess-Newsome, J., Cardenas, S., Austin, B. A., Carlson, J., Gardner, A. L., Stuhlsatz, M. A. M., Wilson, C. D. et al. (2011, April). Impact of educative materials and transformative professional development on teachers’ PCK, practice, and student achievement. Paper presented at the NARST Annual Meeting, Orlando.

  • Grossman, P. L. (1990). The making of a teacher: Teacher knowledge and teacher education. New York: Teachers College Press.

    Google Scholar 

  • Hashweh, M. (2005). Teacher pedagogical constructions: A reconfiguration of PCK. Teachers and Teaching: Theory and Practice, 11, 237–292.

    Article  Google Scholar 

  • Hattie, J. & Cooksey, R. W. (1984). Procedures for assessing the validities of tests using the “known-groups” method. Applied Psychological Measurement, 8, 295–305.

    Article  Google Scholar 

  • Heller, J. L., Daehler, K. R., Shinohara, M., & Kaskowitz, S. R. (2004, April). Fostering pedagogical content knowledge about electric circuits through case-based professional development. Paper presented at the NARST Annual Meeting, Vancouver. Retrieved from http://www.wested.org/understandingscience/downloads/ppr_fostped.pdf.

  • Hill, H. C., Ball, D. L., Blunk, M., Goffney, I. M. & Rowan, B. (2007). Validating the ecological assumption: The relationship of measure scores to classroom teaching and student learning. Measurement: Interdisciplinary Research and Perspectives, 5, 371–406.

    Google Scholar 

  • Hill, H. C., Loewenberg Ball, D. & Schilling, S. G. (2008). Unpacking pedagogical content knowledge: Conceptualizing and measuring teachers’ topic-specific knowledge of students. Journal for Research in Mathematics Education, 38, 372–400.

    Google Scholar 

  • Hill, H. C., Rowan, B. & Ball, D. (2005). Effects of teachers’ mathematical knowledge for teaching on student achievement. American Educational Research Journal, 42, 371–406.

    Article  Google Scholar 

  • Jatzwauk, P., Rumann, S. & Sandmann, A. (2008). Der Einfluss des Aufgabeneinsatzes im Biologieunterricht auf die Lernleistung der Schüler—Ergebnisse einer Videostudie. [The influence of tasks on the students’ learning performance in biology lessons—Results of a video study]. Zeitschrift für Didaktik der Naturwissenschaften, 14, 263–281.

    Google Scholar 

  • Kind, V. (2009). Pedagogical content knowledge in science education: Perspectives and potential for progress. Studies in Science Education, 45, 169–204.

    Article  Google Scholar 

  • Knight, P. (2002). A systemic approach to professional development: Learning as practice. Teaching and Teacher Education, 18, 229–241.

    Article  Google Scholar 

  • Krauss, S., Baumert, J. & Blum, W. (2008). Secondary mathematics teachers’ pedagogical content knowledge and content knowledge: Validation of the COACTIV constructs. The International Journal of Mathematics Education, 40, 873–892.

    Google Scholar 

  • Kromrey, J. D., & Renfrow, D. D. (1991). Using multiple choice examination items to measure teachers’ content specific pedagogical knowledge. Annual Meeting of the Eastern Educational Research Association, Boston. Retrieved from ERIC database (ED329594).

  • Lee, E. & Luft, J. A. (2008). Experienced secondary science teachers’ representation of pedagogical content knowledge. International Journal of Science Education, 30, 1343–1363.

    Article  Google Scholar 

  • Loughran, J., Berry, A. & Mulhall, P. (2006). Understanding and developing science teachers’ pedagogical content knowledge. Rotterdam: Sense.

    Google Scholar 

  • Loughran, J., Milroy, P., Berry, A., Gunstone, R. & Mulhall, P. (2001). Documenting science teachers’ pedagogical content knowledge through PaP-eRs. Research in Science Education, 31, 289–307.

    Article  Google Scholar 

  • Magnusson, S., Krajcik, J. & Borko, H. (1999). Nature, sources, and development of pedagogical content knowledge. In J. Gess-Newsome & N. G. Lederman (Eds.), Examining pedagogical content knowledge (pp. 95–132). Dordrecht: Kluwer Academic.

    Google Scholar 

  • Marks, R. (1990). Pedagogical content knowledge: From a mathematical case to a modified conception. Journal of Teacher Education, 41, 3–11.

    Article  Google Scholar 

  • Mulhall, P., Berry, A., & Loughran, J. (2003). Frameworks for representing science teachers’ pedagogical content knowledge. Asia-Pacific Forum on Science Learning and Teaching, 4(2), Article 2. Retrieved from http://www.ied.edu.hk/apfslt/v4_issue2/mulhall/index.htm.

  • OECD (2000). Measuring student knowledge and skills—The PISA 2000 Assessment of Reading Mathematical and Scientific Literacy. Paris: OECD—Organisation for Economic Co-Operation and Development.

    Google Scholar 

  • Park, S., Jang, J., Chen, Y. & Jung, J. (2011). Is pedagogical content knowledge (PCK) necessary for reformed science teaching? Evidence from an empirical study. Research in Science Education, 41, 254–260.

    Article  Google Scholar 

  • Park, S. & Oliver, S. J. (2008). Revisiting the conceptualisation of pedagogical content knowledge (PCK): PCK as a conceptual tool to understand teachers as professionals. Research in Science Education, 38, 261–284.

    Article  Google Scholar 

  • Piburn, M., & Sawada, D. (2003). Reformed Teaching Observation Protocol (RTOP): Reference manual (rep. no. IN00-3). Retrieved from http://physicsed.buffalostate.edu/AZTEC/RTOP/RTOP_full/.

  • Polanyi, M. (1958). Personal knowledge. Chicago: University of Chicago Press.

    Google Scholar 

  • Rohaan, E. J., Taconis, R. & Jochems, W. M. G. (2009). Measuring teachers’ pedagogical content knowledge in primary biology education. Research in Science and Technological Education, 27, 327–338.

    Article  Google Scholar 

  • Ryle, G. (1971). ‘Knowing how and knowing that’ (Proceedings of the Aristotelian Society, 1946). In G. Ryle (Ed.), Collected papers. Volume II: Collected essays, 1929–1968 (pp. 212–225). London: Hutchinson & Co.

    Google Scholar 

  • Schilling, S. G. & Hill, H. C. (2007). Assessing measures of mathematical knowledge for teaching: A validity argument approach. Measurement: Interdisciplinary Research and Perspectives, 5, 93–106.

    Article  Google Scholar 

  • Schmidt, W. H., Tatto, M. T., Bankov, K., Blömeke, S., Cedillo, T., Cogan, L., Schwille, J. et al. (2007). The preparation gap: Teacher education for middle school mathematics in six countries. MT21 report. East Lansing: Michigan State University.

  • Shrout, P. E. & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420–428.

    Article  Google Scholar 

  • Shulman, L. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15(2), 4–14.

    Article  Google Scholar 

  • Shulman, L. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57, 1–22.

    Google Scholar 

  • Smith, D. C. & Neale, D. C. (1989). The construction of subject matter knowledge in primary science teaching. Teaching and Teacher Education, 5, 1–20.

    Article  Google Scholar 

  • Staub, F. C. & Stern, E. (2002). The nature of teachers’ pedagogical content beliefs matters for students’ achievement gains. Quasi-experimental evidence from elementary mathematics. Journal of Educational Psychology, 94, 344–355.

    Article  Google Scholar 

  • Stillings, N., Weisler, S. E., Chase, C. H., Feinstein, M. H., Garfield, J. L. & Rissland, E. L. (1995). Cognitive science: An introduction. Cambridge: MIT.

    Google Scholar 

  • Sungur, S., Tekkaya, C. & Geban, O. (2001). The contribution of conceptual change texts accompanied by concept mapping students understanding of the human circulatory system. School Science and Mathematics: Official Journal of the School Science and Mathematics Association, 101, 1–14.

    Google Scholar 

  • Tamir, P. (1988). Subject matter and related pedagogical knowledge in teacher education. Teaching and Teacher Education: an Internal Journal of Research and Studies, 4, 99–110.

    Article  Google Scholar 

  • Tepner, O., Borowski, A., Dollny, S., Fischer, H. E., Jüttner, M., Kirschner, S., Wirth, J. et al. (2012). Modell zur Entwicklung von Testitems zur Erfassung des Professionswissens von Lehrkräften in den Naturwissenschaften [Item development model for assessing professional knowledge of science teachers]. Zeitschrift für Didaktik der Naturwissenschaften, 18, 7–28.

  • Tiemann, R., Rumann, S., Jatzwauk, P. & Sandmann, A. (2006). Aufgaben aus Lehrersicht. [Tasks from teachers’ point of view]. Der Mathematische und Naturwissenschaftliche Unterricht, 59, 304–307.

    Google Scholar 

  • Wadouh, J., Sandmann, A. & Neuhaus, B. J. (2009). Vernetzung im Biologieunterricht—deskriptive Befunde einer Videostudie. [Knowledge linking levels in biology lessons—Descriptive results of a video study]. Zeitschrift für Didaktik der Naturwissenschaften, 15, 69–87.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Formerly: Research Group and Graduate School “Teaching and Learning Science”, University of Duisburg–Essen, Essen, Germany

    Stephan Schmelzing & Stefanie Brandenbusch

  2. ICLON—Leiden University Graduate School of Teaching, Leiden, Netherlands

    Jan H. van Driel

  3. Biology Education, Faculty of Biology, Ludwig-Maximilians-University Munich, Winzererstr. 45, 80797, Munich, Germany

    Melanie Jüttner & Birgit J. Neuhaus

  4. Biology Education, University of Duisburg–Essen, Essen, Germany

    Angela Sandmann

Authors
  1. Stephan Schmelzing
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jan H. van Driel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Melanie Jüttner
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Stefanie Brandenbusch
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Angela Sandmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Birgit J. Neuhaus
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Birgit J. Neuhaus.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Schmelzing, S., van Driel, J.H., Jüttner, M. et al. DEVELOPMENT, EVALUATION, AND VALIDATION OF A PAPER-AND-PENCIL TEST FOR MEASURING TWO COMPONENTS OF BIOLOGY TEACHERS’ PEDAGOGICAL CONTENT KNOWLEDGE CONCERNING THE “CARDIOVASCULAR SYSTEM”. Int J of Sci and Math Educ 11, 1369–1390 (2013). https://doi.org/10.1007/s10763-012-9384-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10763-012-9384-6

KEY WORDS

Search

Navigation