Skip to main content
SpringerLink
Log in

Nonuniform DIF Detection using Discriminant Logistic Analysis and Multinomial Logistic Regression: A comparison for polytomous items

  • Published:
Quality and Quantity Aims and scope Submit manuscript

Abstract

This study focused on the effectiveness in nonuniform polytomous item DIF detection using Discriminant Logistic Analysis (DLA) and Multinomial Logistic Regression (MLR). A computer simulation study was conducted to compare the effect of using DLA and MLR, applying either an iterative test purification procedure or non-iterative to detect nonuniform DIF. The conditions under study were: DIF effect size (0.5, 1.0 and 1.5), sample size (500 and 1000), percentage of DIF items in the test (0, 10 and 20%) and DIF type (nonuniform). The results suggest that DLA is more accurate than MLR in detecting DIF. However, the purification process only improved the correct detection rate when MLR was applied. The false positive rates for both procedures were similar. Moreover, when the test purification procedure was used, the proportion of non-DIF items that were detected as DIF decreased for both procedures, although the false positive rates were smaller for DLA than for MLR.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • A. Agresti (1984) Analysis of Ordinal Categorical Data Wiley New York, NY

    Google Scholar 

  • A. Agresti (1990) Categorical Data Analysis Wiley New York, NY

    Google Scholar 

  • Y. M. M. Bishop S. E. Fienberg P. W. Holland (1975) Discrete Multivariate Analysis: Theory and Practice MIT Press Cambridge, MA

    Google Scholar 

  • D. M. Bolt (2002) ArticleTitleA Monte Carlo Comparison of Parametric and Nonparametric Polytomous DIF Detection Methods Applied Measurement in Education 15 113–141 Occurrence Handle10.1207/S15324818AME1502_01

    Article  Google Scholar 

  • H.H. Chang J. Mazzeo L. Roussos (1996) ArticleTitleDetecting DIF for polytomously scored items: An adaptation of SIBTEST procedure Journal of Educational Measurement 33 333–353 Occurrence Handle10.1111/j.1745-3984.1996.tb00496.x

    Article  Google Scholar 

  • B. Clauser K. M. Mazor R. K. Hambleton (1993) ArticleTitleThe effects of purification of the matching criterion on the identification of DIF using the Mantel-Haenszel procedure Applied Measurement in Education 6 269–279 Occurrence Handle10.1207/s15324818ame0604_2

    Article  Google Scholar 

  • A. S. Cohen S. H. Kim F. B. Baker (1993) ArticleTitleDetection of differential item functioning in the graded response model Applied Psychological Measurement 17 335–350

    Google Scholar 

  • A. S. Cohen S. H. Kim J. A. Wollack (1996) ArticleTitleAn investigation of the likelihood ratio test for detection of differential item functioning Applied Psychological Measurement 20 15–26

    Google Scholar 

  • De Ayala, R. J., Kim, S. -H., Stapleton, L. M., and Dayton, C. M. (1999). A Reconceptualization of Differential Item Functioning. Paper presented at the Annual Meeting of the American Educational Research Association. Montreal, Canada.

  • B. Ellis (1989) ArticleTitleDifferential item functioning: Implications for test translations Journal of Applied Psychology 74 912–921 Occurrence Handle10.1037/0021-9010.74.6.912

    Article  Google Scholar 

  • C. P. Flowers T. C. Oshima N. S. Raju (1999) ArticleTitleA description and demonstration of the Polytomous-DFIT framework Applied Psychological Measurement 23 309–326 Occurrence Handle10.1177/01466219922031437

    Article  Google Scholar 

  • A. W. French T. R. Miller (1996) ArticleTitleLogistic Regression and its use in detecting differential item functioning in polytomous items Journal of Educational Measurement 33 315–332 Occurrence Handle10.1111/j.1745-3984.1996.tb00495.x

    Article  Google Scholar 

  • GAUSS System version 3.0 (1992). Washington: Aptech Systems, Inc.

  • R. K. Hambleton L. Cook (1983) Robustness of Item Response models and effects of test length and sample size on the precision of ability estimates D. J. Weiss (Eds) New Horizons in Testing: Latent Trait Test Theory and Computerized Adaptative Testing Academic Press New York

    Google Scholar 

  • R. K. Hambleton H. J. Rogers (1989) ArticleTitleDetecting potentially biased test items: comparison of IRT area and Mantel-Haenszel methods Applied Measurement in Education 2 313–334 Occurrence Handle10.1207/s15324818ame0204_4

    Article  Google Scholar 

  • M. D. Hidalgo J. Gómez (2003) ArticleTitleTest purification and the evaluation of differential item functioning with multinomial logistic regression European Journal of Psychological Assessment 19 1–11 Occurrence Handle10.1027//1015-5759.19.1.1

    Article  Google Scholar 

  • G. J. Lautenschlager V. L. Flaherty D. Park (1994) ArticleTitleIRT differential item functioning: An examination of ability scale purification Educational and Psychological Measurement 54 21–31

    Google Scholar 

  • G. J. Mellenbergh (1982) ArticleTitleContingency table models for assessing items bias Journal of Educational Statistics 7 105–118 Occurrence Handle10.2307/1164960

    Article  Google Scholar 

  • S. Menard (1995) Applied Logistic Regression Analysis. SAGE Paper series on Quantitative Applications in the Social Sciences, 07-106 Sage Thousand Oaks, CA

    Google Scholar 

  • M. D. Miller T. C. Oshima (1992) ArticleTitleEffect of sample size, number of biased items and magnitude of bias on a two-stage item bias estimation method Applied Psychological Measurement 16 381–388

    Google Scholar 

  • T. R. Miller J. A. Spray (1993) ArticleTitleLogistic discriminant function analysis for DIF identification of polytomously scored items Journal of Educational Measurement 30 107–122 Occurrence Handle10.1111/j.1745-3984.1993.tb01069.x

    Article  Google Scholar 

  • Miller, T. R., Spray, J. A., and Wilson, A. (1992). A comparison of three methods for identifying nonuniform DIF in polytomously scored test items. Paper presented at the Psychometric Society Meeting, Columbus, OH.

  • R. E. Millsap H. T. Everson (1993) ArticleTitleMethodology Review: Statistical approaches for assessing measurement bias Applied Psychological Measurement 16 389–402

    Google Scholar 

  • P. Narayanan H. Swaminathan (1996) ArticleTitleIdentification of items than show nonuniform DIF Applied Psychological Measurement 20 257–274

    Google Scholar 

  • R. D. Penfield T. C. M. Lam (2000) ArticleTitleAssessing differential item functioning in performance assessment: Review and recommendations Educational Measurement: Issues and Practice 19 5–15 Occurrence Handle10.1111/j.1745-3992.2000.tb00033.x

    Article  Google Scholar 

  • M.T. Potenza N.J. Dorans (1995) ArticleTitleDIF assessment for polytomously scored items: a framework for classification and evaluation Applied Psychological Measurement 19 23–37

    Google Scholar 

  • H. J. Rogers H. Swaminathan (1993) ArticleTitleA comparison of Logistic regression and Mantel-Haenszel procedures for detecting differential item functioning Applied Psychological Measurement 17 105–116

    Google Scholar 

  • Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometric Monograph Supplement 17.

  • SAS Institute. (1986). SAS User’s guide (Release 6.04). SAS Institute, Cary NC

  • H. Swaminathan H. J. Rogers (1990) ArticleTitleDetecting differential item functioning using logistic regression procedures Journal of Educational Measurement 27 361–370 Occurrence Handle10.1111/j.1745-3984.1990.tb00754.x

    Article  Google Scholar 

  • Zumbo, B. D. (1999). A Handbook on the theory and methods of differential item functioning(DIF): Logistic Regression Modeling as a unitary framework for binary and Likert-type ordinal) item scores. Ottawa, ON: Directorate of Human Resources Research and Evaluation, Department of National Defense.

Download references

Author information

Authors and Affiliations

  1. Dpto. Psicología Básica y Metodología, Facultad de Psicología, Universidad de Murcia, Campus de Espinardo, Apartado 4021, 30080, Murcia, Spain

    M. Dolores Hidalgo

  2. Dpto. Metodología de las Ciencias del Comportamiento, Facultad de Psicología, Universitat de Barcelona, Passeig Vall d’Hebrón, 171, 08035, Barcelona, Spain

    Juana Gómez

Authors
  1. M. Dolores Hidalgo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Juana Gómez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. Dolores Hidalgo.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Hidalgo, M.D., Gómez, J. Nonuniform DIF Detection using Discriminant Logistic Analysis and Multinomial Logistic Regression: A comparison for polytomous items. Qual Quant 40, 805–823 (2006). https://doi.org/10.1007/s11135-005-3964-2

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11135-005-3964-2

Keywords

Search

Navigation