Skip to main content
Log in

Measuring the degree of implementation of school-based assessment schemes for practical science

  • Published:
Research in Science Education Aims and scope Submit manuscript

Abstract

The study described in this paper investigated the degree of implementation of school-based assessment schemes for practical science. An assessment scheme in Hong Kong was used as the context for the research. Five dimensions of implementation were identified and a conceptual framework was built by breaking down each dimension into attributes and these into content areas. The research method used in the study incorporated some modifications on most past designs in order to determine adequately the construct validity of data and to examine any method-specific biases. Three measuring instruments (namely, a student questionnaire, a teacher questionnaire and an observation schedule) were designed. The construct validity of student data was demonstrated through confirmatory factor analysis and multitrait-multimethod analysis. Comparative analyses of students' perceptions, teacher self-reports and independent observational data resulted in convergent findings for only three out of the ten attributes of implementation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Ames, R. (1983). Teachers' attributions for their own teaching. In J. M. Levine & M. C. Wang (Eds.),Teacher and student perceptions: Implications for learning (pp. 105–124). Hillsdale, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Baker, F. B. (1992). Equating tests under the graded response model.Applied Psychological Measurement, 16(1), 87–96.

    Google Scholar 

  • Baker, F. B. (1994).EQUATE computer program. University of Wisconsin: Department of Educational Psychology.

  • Bowen, G., & Lock, R. (1990). Assessing practical skills at A-level.Education in Chemistry, 27(5), 137–138.

    Google Scholar 

  • Buchan, A. S. & Jenkins, E. W. (1992). The internal assessment of practical skills in science in England and Wales, 1960–1991: Some issues in historical perspective.International Journal of Science Education, 14(4), 367–380.

    Google Scholar 

  • Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix.Psychological Bulletin, 56, 81–105.

    Google Scholar 

  • Cheung, D. (1995).Measurement of the degree of implementation of school-based assessment schemes for practical science. Unpublished PhD thesis, University of Western Australia.

  • Cheung, D., Hattie, J., Bucat, R., & Douglas, G. (in press). Identifying the dimensions of curriculum implementation: The nominal group technique.Curriculum Forum.

  • Clark, C. M., & Peterson, P. L. (1986). Teachers' thought processes. In M. C. Wittrock (Ed.),Handbook of research on teaching (pp. 255–296). New York: Macmillan.

    Google Scholar 

  • Cohen, J. (1960). A coefficient of agreement for nominal scales.Educational and Psychological Measurement, 20(1), 37–46.

    Google Scholar 

  • Fullan, M., & Pomfret, A. (1977). Research on curriculum and instruction implementation.Review of Educational Research, 47(1), 335–397.

    Google Scholar 

  • Hall, G. E., & Hord, S. M. (1987).Change in schools: Facilitating the process. Albany, NY: State University of New York Press.

    Google Scholar 

  • Hong Kong Education Department. (1990). Evaluation of the teacher assessment scheme for advanced level chemistry practical.School Science Newsletter, 30, 38–43.

    Google Scholar 

  • Hong Kong Examinations Authority. (1991).Handbook on the teacher assessment scheme for practical chemistry. Hong Kong: Hong Kong Examinations Authority.

    Google Scholar 

  • Jenkins, E. W. (1992). School-based assessment of practical competence in science: Some issues from England experience.Research in Science Education, 22, 224–229.

    Google Scholar 

  • Kempa, R. (1986).Assessment in science. Cambridge: Cambridge University Press.

    Google Scholar 

  • Krakowski K., & Hattie J. A. (1994).MICFA: A Macintosh interactive program for confirmatory factor analysis. Perth: Graduate School of Education, University of Western Australia.

    Google Scholar 

  • Leithwood, K. A., & Montgomery, D. J. (Eds.). (1987).Improving classroom practice using innovation profiles. Toronto: The Ontario Institute for Studies in Education.

    Google Scholar 

  • Marsh, H. W., Balla, J. R., & McDonald, R. P. (1988). Goodness-of-fit indexes in confirmatory factor analysis: The effect of sample size.Psychological Bulletin, 103(3), 391–410.

    Article  Google Scholar 

  • Mathison, S. (1988). Why triangulate?Educational Researcher, 17(2), 13–17.

    Google Scholar 

  • Pang, K. C. (1992). The biology teacher assessment scheme (TAS).Curriculum Forum, 2, 81–90.

    Google Scholar 

  • Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores.Psychometrika Monography Supplement,4 (Part 2, Whole #17).

  • Scheirer, M. A., & Rezmovic, E. L. (1983). Measuring the degree of program implementation: A methodological review.Evaluation Review, 7(5), 599–633.

    Google Scholar 

  • Snyder, J., Bolin, F., & Zumwalt, K. (1992). Curriculum implementation. In P. W. Jackson (Ed.),Handbook of research on curriculum (pp. 402–435). New York: Macmillan.

    Google Scholar 

  • Stein, M. K., & Wang, M. C. (1988). Teacher development and school improvement: The process of teacher change.Teaching & Teacher Education, 4(2), 171–187.

    Article  Google Scholar 

  • Thissen, D. (1991).MULTILOG user's guide: Multiple, categorical item analysis and test scoring using item response theory. Chicago: Scientific Software.

    Google Scholar 

  • Yin, R. K. (1982). Studying the implementation of public programs. In W. Williams (Ed.),Studying implementation: Methodological and administrative issues (pp. 36–72). Chatham, NJ: Chatham House.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Derek Cheung.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Cheung, D., Hattie, J., Bucat, R. et al. Measuring the degree of implementation of school-based assessment schemes for practical science. Research in Science Education 26, 375–389 (1996). https://doi.org/10.1007/BF02357450

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02357450

Keywords

Navigation