Skip to main content
SpringerLink
Log in

Modelling mathematics problem solving item responses using a multidimensional IRT model

  • Articles
  • Published:
Mathematics Education Research Journal Aims and scope Submit manuscript

Abstract

This research examined students’ responses to mathematics problem-solving tasks and applied a general multidimensional IRT model at the response category level. In doing so, cognitive processes were identified and modelled through item response modelling to extract more information than would be provided using conventional practices in scoring items. More specifically, the study consisted of two parts. The first part involved the development of a mathematics problem-solving framework that was theoretically grounded, drawing upon research in mathematics education and cognitive psychology. The framework was then used as the basis for item development. The second part of the research involved the analysis of the item response data. It was demonstrated that multidimensional IRT models were powerful tools for extracting information from a limited number of item responses. A problem-solving profile for each student could be constructed from the results of IRT scaling.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Adams, R. J., Wilson, M. R., & Wang, W. (1997). The multidimensional random coefficients multinomial logit model.Applied Psychological Measurement, 21, 1–23.

    Article  Google Scholar 

  • Adams, R. J., & Wu, M. L. (2002).PISA 2000 technical report. Paris: OECD.

    Google Scholar 

  • Bond, T.G., & Fox, C. M. (2007)Applying the Rasch model: Fundamental measurement in the human sciences (2nd ed). Mahwah, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Bonotto, C. (2003). Suspension of sense-making in mathematical word problem solving: A possible remedy. Retrieved August 16, 2003, from http://math.unipa.it/~grim/Jbonotto

  • Cai, J., & Silver, E. A. (1995). Solution processes and interpretations of solutions in solving division-with-remainder story problems: Do Chinese and U.S. students have similar difficulties?Journal for Research in Mathematics Education, 26, 491–497.

    Article  Google Scholar 

  • Carroll, J. B. (1945). The effect of difficulty and chance success on correlations between items or between tests.Psychometrika, 10, 1–19.

    Article  Google Scholar 

  • Carroll, J. B. (1993).Human cognitive abilities. A survey of factor-analytic studies. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  • Carroll, J. B. (1996). Mathematical abilities: Some results from factor analysis. In R. J. Sternberg & B.-Z. Talia (Eds.),The nature of mathematical thinking. Mahwah, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Collis, K. F., & Romberg, T. A. (1992).Collis-Romberg mathematical problem solving profiles. Melbourne: Australian Council for Educational Research.

    Google Scholar 

  • Cornish, G., & Wines, R. (1977).Mathematics profiles series: Operations test teachers handbook. Melbourne: Australian Council for Educational Research.

    Google Scholar 

  • De Lange, J. (1996). Using and applying mathematics in education. In A. J. Bishop, K. Clements, C. Keitel, J. Kilpatrick, & C. Laborde (Eds.),International handbook of mathematics education (pp. 49–98). Dordrecht, The Netherlands: Kluwer.

    Google Scholar 

  • Ellerton, N. F., & Clarkson, P. C. (1996). Language factors in mathematics teaching and learning. In A. J. Bishop, K. Clements, C. Keitel, J. Kilpatrick, & C. Laborde (Eds.),International handbook of mathematics education (pp.987–1033). Dordrecht, The Netherlands: Kluwer.

    Google Scholar 

  • Embretsons, S. E. (1991). A multidimensional latent trait model for measuring learning and change.Psychometrika, 56, 495–515.

    Article  Google Scholar 

  • Embretson, S. E. (1997). Multicomponent response models. In W. J. van der Linden & R. K. Hambleton (Eds.),Handbook of modern item response theory. New York: Springer-Verlag

    Google Scholar 

  • Fredriksen, J., Mislevy, R. J., & Bejar, I. (Eds.) (1991).Test theory for a new generation of tests. Hillsdale, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Gravemeijer, K. (1999). How emergent models may foster the constitution of formal mathematics.Mathematical Thinking and Learning: An International Journal, 1(2), 155–177.

    Article  Google Scholar 

  • Hambleton, R. K., & Rovinelli, R. J. (1986). Assessing the dimensionality of a set of test items.Applied Psychological Measurement, 10, 287–302.

    Article  Google Scholar 

  • Heim, A. W. (1975).Psychological testing. London: Oxford University Press.

    Google Scholar 

  • Jöreskog, K. G., & Sörbom, D. (1979).Advances in factor analysis and structural equation models. Cambridge, MA: Abt.

    Google Scholar 

  • Lincare, J. M. (1998). Detecting multidimensionality: Which residual data-type works best?Journal of Outcome Measurement, 2(3), 266–283.

    Google Scholar 

  • Malone, J. A., Douglas, G. A., Kissane, B. V., & Mortlock, R. S. (1980). Measuring problem-solving ability. In S. Krulik & R. E. Reys (Eds.),Problem solving in school mathematics (pp. 204–215). Reston, VA: NCTM.

    Google Scholar 

  • Masters, G. N. (1982). A Rasch model for partial credit scoring.Psychometrika, 47, 149–174.

    Article  Google Scholar 

  • Masters, G. N. & Doig, B. A. (1992). Understanding children’s mathematics: Some assessment tools. In G. Leder (Ed.),Assessment and learning of mathematics. Melbourne: Australian Council of Educational Research.

    Google Scholar 

  • Mayer, R. E., & Hegarty, M. (1996). In R. J. Sternberg & B.-Z. Talia (Eds.),The nature of mathematical thinking. Mahwah, NJ: Lawrence Erlbaum.

    Google Scholar 

  • McDonald, R. P., & Ahlawat, K. S. (1974). Difficulty factors in binary data.British Journal of Mathematical and Statistical Psychology, 27, 82–99.

    Google Scholar 

  • Nandakumar, R. (1994). Assessing latent trait unidimensionality of a set of items — Comparison of different approaches.Journal of Educational Measurements, 31, 1–18.

    Google Scholar 

  • National Council of Teachers of Mathematics. (1989).Curriculum and evaluation standards for school mathematics. Reston, VA: NCTM.

    Google Scholar 

  • Nesher, P. (1980). The stereotyped nature of school word problems.For the Learning of Mathematics, 1(1), 41–48.

    Google Scholar 

  • Newman, M. A. (1977). An analysis of sixth-grade pupils’ errors on written mathematical tasks.Victorian Institute for Educational Research Bulletin, 39, 31–43.

    Google Scholar 

  • Newman, M. A. (1983).Strategies for diagnosis and remediation. Sydney: Harcourt, Brace Jovanovich.

    Google Scholar 

  • OECD (2003).The PISA 2003 assessment framework. Paris: OECD.

    Google Scholar 

  • Polya, G. (1973).How to solve it: A new aspect of mathematical method. Princeton, NJ: Princeton University Press.

    Google Scholar 

  • Rasch, G. (1960).Probabilistic models for some intelligence and attainment tests. Copenhagen, Denmark: Danish Institute for Educational Research.

    Google Scholar 

  • Romberg, T., & de Lange, J. (1998).Mathematics in context. Chicago: Britannica Mathematics System.

    Google Scholar 

  • Schoenfeld, A. H. (1983). Episodes and executive decisions in mathematical problem solving. In R. Lesh & M. Landau, M. (Eds.),Acquisition of mathematics concepts and processes (pp. 345–395). New York: Academic.

    Google Scholar 

  • Schoenfeld, A. H. (1991). On mathematics as sense-making: An informal attack on the unfortunate divorce of formal and informal mathematics. In J. E. Voss, D. N. Perkins, & J. W. Segal (Eds.),Informal reasoning and education (pp. 311–343). Hillsdale, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Silver, E. A. (1982). Knowledge organisation and mathematical problem solving. In F. K. Lester & J. Garafalo (Eds.),Mathematical problem solving: Issues in research. Philadelphia, PA: Franklin Institute Press.

    Google Scholar 

  • Smith, R. M., & Miao, C. Y. (1994). Assessing unidimensionality for Rasch measurement. In M. Wilson (Ed.),Objective measurement: Theory into practice (Vol. 2, pp. 316–327) Norwood, NJ: Ablex.

    Google Scholar 

  • Stacey, K., Groves, S., Bourke, S., & Doig, B. (1993).Profiles of problem solving. Melbourne: Australian Council for Educational Research.

    Google Scholar 

  • Treffers, A. (1986).Three dimensions. Dordrecht, The Netherlands: Reidel.

    Google Scholar 

  • Verhelst, N. D., Glas, C. A. W., & de Vries, H. H. (1997). A steps model to analyze partial credit. In W. J. van der Linden & R. K. Hambleton (Eds.),Handbook of modern item response theory. New York: Springer-Verlag.

    Google Scholar 

  • Verhelst, N. D. (2001).Some thoughts on reliability. Unpublished manuscript.

  • Verschaffel, L., Greer, B. & de Corte E. (2000). Making sense of word problems. Lisse, Switzerland: Swets & Zeitlinger.

    Google Scholar 

  • Wang, W. (1998). Rasch analysis of distractors in multiple-choice items.Journal of Outcome Measurement, 2(1), 43–65.

    Google Scholar 

  • Whimbey, A., & Lochhead, J. (1991).Problem solving and comprehension (5th ed.) Hillsdale, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Willmott, A. S., & Fowles, D. E. (1974).The objective interpretation of test performance. Windsor, UK: National Foundation for Educational Research Publishing.

    Google Scholar 

  • Linacre, M. J., & Wright, B. D. (2000). WINSTEPS Rasch measurement computer program [Computer software]., Chicago: MESA Press.

    Google Scholar 

  • Wright, B. D., & Masters, G. N. (1982).Rating scale analysis. Chicago: MESA Press.

    Google Scholar 

  • Wu, M. L. (2004).The application of item response theory to measure problem-solving proficiencies. Unpublished doctoral dissertation, The University of Melbourne.

  • Wu, M. L., Adams, R. J., & Wilson, M. R. (1998). ConQuest: Multi-aspect test software [Computer software]. Melbourne: Australian Council for Educational Research.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Assessment Research Centre, University of Melbourne, 3010, Parkville, VIC, Australia

    Margaret Wu & Raymond Adams

Authors
  1. Margaret Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Raymond Adams
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wu, M., Adams, R. Modelling mathematics problem solving item responses using a multidimensional IRT model. Math Ed Res J 18, 93–113 (2006). https://doi.org/10.1007/BF03217438

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF03217438

Keywords

Search

Navigation