Abstract
In this paper we discuss complexities of measurement that can arise in a multidimensional situation. All of the complexities that can occur in a unidimensional situation, such as polytomous response formats, item dependence effects, and the modeling of rater effects such as harshness and variability, can occur, with a correspondingly greater degree of complexity, in the multidimensional case also. However, we will eschew these, and concentrate on issues that arise due to the inherent multidimensionality of the situation. First, we discuss the motivations for multidimensional measurement models, and illustrate them in the context of a state-wide science assessment involving both multiple choice items and performance tasks. We then describe the multidimensional measurement model (NMRCML). This multidimensional model is then applied to the science assessment data set to illustrate two issues that arise in multidimensional measurement. The first issue is the question of whether one should (or perhaps, can) design items that relate to multiple dimensions. The second issue arises when there is more than one form of multidimensionality present in the item design: should one use just one of these dimensionalities, or some, or all? We conclude by discussing further issues yet to be addressed in the area of multidimensional measurement.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
8. References
Ackerman, T. A. (1992). A didactic explanation of item bias, item impact, and item validity from a multidimensional perspective. Journal of Educational Measurement, 29, 67–91.
Adams, R. J., & Wilson, M. (1996). Formulating the Rasch model as a mixed coefficients multinomial logit. In G. Engelhard and M. Wilson, (Eds.), Objective measurement: Theory into Practice. Vol III. Norwood, NJ: Ablex.
Adams, R. J., & Wilson, M. (1996, April). Multi-level modeling of complex item responses in multiple dimensions: Why bother? Paper presented at the annual meeting of the American Educational Research Association, New York.
Adams, R. J., Wilson, M., & Wang, W. (1997). The multidimensional random coefficients multinomial logit model. Applied Psychological Measurement, 21(1), 1–23.
Akaike, H. (1977). On entropy maximisation principle. In P. R. Krischnaiah (Ed.), Applications of statistics. New York: North Holland.
Andersen, E. B. (1985). Estimating latent correlations between repeated testing. Psychometrika, 50, 3–16.
Ansley, T. N., & Forsyth, R. A. (1985). An examination of the characteristics of unidimensional IRT parameter estimates derived from two-dimensional data. Applied Psychological Measurement, 9, 37–48.
Briggs, D. & Wilson, M. (2003). An introduction to multidimensional measurement using Rasch models. Journal of Applied Measurement, 4(1), 87–100.
California Department of Education. (1995). A sampler of science assessment. Sacramento, CA: Author.
Camilli, G. (1992). A conceptual analysis of differential item functioning in terms of a multidimensional item response model. Applied Psychological Measurement, 16, 129–147.
Davey, T. & Hirsch, T. M. (1991). Concurrent and consecutive estimates of examine ability profiles. Paper presented at the Annual Meeting of the Psychometric Society, New Brunswick, NJ.
Embretson, S. E. (1991). A multidimensional latent trait model for measuring learning and change. Psychometrika, 56, 495–515.
Folk, V. G., & Green, B. F. (1989). Adaptive estimation when the unidimensionality assumption of IRT is violated. Applied Psychological Measurement, 13, 373–389.
Glas, C.A.W. (1989). Contributions to Estimating and Testing Rasch Models. Doctoral Dissertation. Univeristy of Twente
Glas, C. A. W. (1992). A Rasch model with a multivariate distribution of ability. in M. Wilson (Ed). Objective measurement: Theory into Practice. Vol. 1. Norwood, NJ: Ablex Publishing Corporation.
Hambleton, R. K., & Murray, L. N. (1983). Some goodness of fit investigations for item response models. In R. K. Hambleton (Ed.), Applications of item response theory, Vancouver BC: Educational Research Institute of British Columbia.
Kelderman, H., & Rijkes, C. P. M. (1994). Loglinear multidimensional IRT models for polytomously scored items. Psychometrika, 59, 149–176.
Lord, F. M. (1980). Applications of Item Response Theory to Practical Testing Problems. Hillsdale, NJ: Erlbaum.
Luecht, R. M., & Miller, R. (1992). Unidimensional calibrations and interpretations of composite traits for multidimensional tests. Applied Psychological Measurement, 16,3, 279–293.
Muthen, B. (1984). A general structural equation model with dichotomous, ordered categorical and continuous latent variable indicators. Psychometrika, 49, 115–132.
Muthen, B. (1987). LISCOMIP: Analysis of linear structural equations using a comprehensive measurement model. User’s guide. Mooresville, IN: Scientific Software.
Oshima, T. C., & Miller, M. D. (1992). Multidimensionality and item bias in item response theory. Applied Psychological Measurement, 16,3, 237–248.
Rasch, G. (1960, 1980). Probabilistic Models for Some Intelligent and Attainment Tests. Copenhagen: Danmarks Paedogogiske Institut.
Reckase, M. D. (1985). The difficulty of test items that measure more than one ability. Applied Psychological Measurement, 9. 401–412.
Reckase, M. D., & McKinley, R. L. (1991). The discriminating power of items that measure more than one dimension. Applied Psychological Measurement, 15, 361–373.
Wang, W. (1994). Implementation and application of the multidimensional random coefficients multinomial logit. Unpublished doctoral dissertation. University of California, Berkeley.
Wang, W., & Wilson, M. (1996). Comparing open-ended items and performance-based items using item response modeling. In, G. Engelhard & M. Wilson (Eds.), Objective measurement III: Theory into practice. Norwood, NJ: Ablex.
Way, W. D., Ansley, T. N., & Forsyth, R. A. (1988). The comparative effects of compensatory and non-compensatory two-dimensional data on unidimensional IRT estimation. Applied Psychological Measurement, 12, 239–252.
Wilson, M. (1994). Community of judgement: A teacher-centered approach to educational accountability. In, Office of Technology Assessment (Ed.), Issues in Educational Accountability. Washington, D.C.: Office of Technology Assessment, United States Congress.
Wilson, M., & Adams, R. J. (1995). Rasch models for item bundles. Psychometrika, 60, 181–198.
Wilson, M., & Adams R.J. (1996). Evaluating progress with alternative assessments: A model for Chapter 1. In M.B. Katie (Ed,), Implementing performance assessment: Promise, problems and challenges. Hillsdale, NJ: Erlbaum.
Wilson, M.R. & Wang, W.C. (1995). Complex composites: Issues that arise in combining different models of assessment. Applied Psychological Measurement, 19(1), 51–72.
Wu, M., Adams, R.J., & Wilson, M. (1998). ACER ConQuest [computer program]. Hawthorn, Australia: ACER.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer
About this chapter
Cite this chapter
Wilson, M., Hoskens, M. (2005). Multidimensional Item Responses: Multimethod-Multitrait Perspectives. In: Maclean, R., et al. Applied Rasch Measurement: A Book of Exemplars. Education in the Asia-Pacific Region: Issues, Concerns and Prospects, vol 4. Springer, Dordrecht. https://doi.org/10.1007/1-4020-3076-2_16
Download citation
DOI: https://doi.org/10.1007/1-4020-3076-2_16
Publisher Name: Springer, Dordrecht
Print ISBN: 978-1-4020-3072-7
Online ISBN: 978-1-4020-3076-5
eBook Packages: Humanities, Social Sciences and LawEducation (R0)