Abstract
Various different item response theory (IRT) models can be used in educational and psychological measurement to analyze test data. One of the major drawbacks of these models is that efficient parameter estimation can only be achieved with very large data sets. Therefore, it is often worthwhile to search for designs of the test data that in some way will optimize the parameter estimates. The results from the statistical theory on optimal design can be applied for efficient estimation of the parameters.
A major problem in finding an optimal design for IRT models is that the designs are only optimal for a given set of parameters, that is, they are locally optimal. Locally optimal designs can be constructed with a sequential design procedure. In this paper minimax designs are proposed for IRT models to overcome the problem of local optimality. Minimax designs are compared to sequentially constructed designs for the two parameter logistic model and the results show that minimax design can be nearly as efficient as sequentially constructed designs.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Adema, J. J. (1990).Models and algorithms for the construction of achievement tests. Unpublished doctoral dissertation, University of Twente.
Andrich, D. (1978). A rating scale formulation for ordered response categories.Psychometrika, 43, 561–573.
Atkinson, A. C. & Donev, A. N. (1996).Optimum experimental designs. Oxford: Clarendon Press.
Bartholomew, D. J. (1987).Latent variable models and factor analysis. London: Oxford University Press.
Berger, M. P. F. (1992). Sequential sampling designs for the two-parameters item response theory model.Psychometrika, 57, 521–538.
Berger, M. P. F. (1994a). D-optimal sequential sampling designs for item response theory models.Journal of Educational Statistics, 19, 43–56.
Berger, M. P. F. (1994b). A general approach to algorithmic design of fixed-form tests, adaptive tests and testlets.Applied Psychological Measurement, 18, 141–153.
Berger, M. P. F. & Mathijssen, E. (1997). Optimal test designs for polytomously scored items.British Journal of Mathematical and Statistical Psychology, 50, 127–141.
Berger, M. P. F. & Veerkamp, W. J. J. (1996). A review of selection methods for optimal test design. In M. Wilson and G. Engelhard (Eds.),Objective measurement: theory into practice, Volume III (pp. 437–455), Norwoord, NJ, Ablex Publishing.
Boekkooi-Timminga, E. (1989).Models for computerized test construction, Unpublished doctoral dissertation, University of Twente.
Bock, R.D. (1972) Estimating item parameters and latent ability when responses are scored in two or more nominal categories.Psychometrika, 37, 29–51.
Chaloner, K. & Larntz, K. (1989). Optimal Bayesian design applied to logistic regression experiments.Journal of Statistical Planning and Inference, 21, 191–208.
Fedorov, V.V. (1980). Convex design theory.Mathematische Operations forschung und Statistics, Serie Statistics, 11, 403–413.
Ford, I., Kitsos, C. P., & Titterington, D. M. (1989). Recent advances in nonlinear experimental design.Technometrics, 31, 49–60.
Heinen, A. G. J. J. (1996).Latent class and discrete latent variable models. London: Sage Publications.
Jones, D. H. & Jin, Z. (1994). Optimal sequential designs for on-line item estimation.Psychometrika, 59, 59–75.
Kiefer, J. (1974). General equivalence theory for optimum designs (Approximate theory).The Annals of Statistics, 2, 849–879.
Kiefer, J. & Wolfowitz (1960). The equivalence of two extremum problems.Canadian Journal of Mathematics, 12, 363–366.
King, C. Y. Joy (1996)Minimax optimal designs. Unpublished doctoral disseration, UCLA Department of Biostatistics.
Lord, F. M. (1980).Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.
Masters, G. N. (1982). A Rasch model for partial credit scoring.Psychometrika, 47, 149–174.
Parthasarathy, K. R. (1967).Probability measures on metric spaces. New York: Academic Press.
Pazman (1986).Foundations of optimum experimental design. Dordrecht/Boston/Lancaster: D. Reidel Publishing.
Pshenichnyi, B. N. (1971).Necessary conditions for an extremum. New York: Marcel Dekker.
Sitter, R. R. (1992). Robust designs for binary data.Biometrics, 48, 1145–1155.
Stefanski, L. A. and Carroll, R. J. (1985). Covariate measurement error in logistic regression.Annals of Statistics, 13, 1335–1351.
Silvey, S. D. (1980).Optimal design. London: Chapman & Hall.
Stocking, M. L. (1990). Specifying optimum examinees for item parameter estimation in item response theory.Psychometrika, 55, 461–475.
Theunissen, T.J.J.M. (1985). Binary programming and test design.Psychometrika, 50, 411–420.
Thissen, D. & Steinberg, L. (1986). A taxonomy of item response models.Psychometrika, 51, 567–577.
Thissen, D. & Wainer, H. (1982). Some standard errors in item response theory.Psychometrika, 47, 397–412.
Van der Linden, W. J. & Boekkooi-Timminga, E. (1989). A maximin model for test design with practical constraints.Psychometrika, 53, 237–247.
Wingersky, M. S. & Lord, F. M. (1984). An investigation of methods for reducing sampling error in certain IRT procedures.Applied Psychological Measurement, 8, 347–364.
Wong, W. K. (1992). A unified approach to the construction of minimax designs.Biometrika, 79, 611–619.
Wong, W. K., & Cook, R. D. (1993). Heteroscedastic G-optimal designs.Journal of the royal Statistical Society, Series B, 55, 871–880.
Wynn, H. P. (1970). The sequential generation of D-optimum experimental designs.Annals of Mathematical Statistics, 41, 1655–1664.
Wu, C.F.J. (1985) Efficient sequential designs with binary data.Journal of the American Statistical Association, 80, 974–984.
Zhu, W., Ahn, H. & Wong, W. K. (1998). Multiple objective optimal designs for the logit model.Communications in Statistics (Theory and Methods), 27, 1581–1592.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Berger, M.P.F., Joy King, C.Y. & Wong, W.K. Minimax d-optimal designs for item response theory models. Psychometrika 65, 377–390 (2000). https://doi.org/10.1007/BF02296152
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF02296152