Abstract
Purpose
To compare the results from a simulated computerized adaptive test (CAT) for the 28-item Impact of Vision Impairment (IVI) questionnaire and the original paper–pencil version in terms of efficiency (main outcome), defined as percentage item reduction.
Methods
Using paper–pencil IVI data from 832 participants across the spectrum of vision impairment, item calibrations of the 28-item IVI instrument and its associated 20-item vision-specific functioning (VSF) and 8-item emotional well-being (EWB) subscales were generated with Rasch analysis. Based on these calibrations, CAT simulations were conducted on 1000 cases, with ‘high’ and ‘moderate’ precision stopping rules (standard error of measurement [SEM] 0.387 and 0.521, respectively). We examined the average number of items needed to satisfy the stopping rules and the corresponding percentage item reduction, level of agreement between person measures estimated from the full IVI item bank and from the CAT simulations, and item exposure rates (IER).
Results
For the overall IVI-CAT, 5 or 9.7 items were required, on average, to obtain moderate or high precision estimates of vision-related quality of life, corresponding to 82.1 and 65.4% item reductions compared to the paper–pencil IVI. Agreement was high between the person measures generated from the full IVI item bank and the IVI-CAT for both the high precision simulation (mean bias, − 0.004 logits; 95% LOA − 0.594 to 0.587) and moderate precision simulation (mean bias, 0.014 logits; 95% LOA − 0.828 to 0.855). The IER for the IVI-CAT in the moderate precision simulation was skewed, with six EWB items used > 40% of the time.
Conclusion
Compared to the paper–pencil IVI instrument, the IVI-CATs required fewer items without loss of measurement precision, making them potentially attractive outcome instruments for implementation into clinical trials, healthcare, and research. Final versions of the IVI-CATs are available.
Similar content being viewed by others
Data availability
The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.
References
Fenwick, E. K., Man, R. E., Ong, P. G., Sabanayagam, C., Gupta, P., Cheng, C. Y., et al. (2016). Association of changes in visual acuity with vision-specific functioning in the Singapore Malay Eye Study. JAMA Ophthalmology,134(11), 1299–1305.
Fenwick, E., Ong, P., Man, R., Cheng, C.-Y., Sabanayagam, C., Wong, T., et al. (2016). Association of vision impairment and major eye diseases with mobility and independence in a Chinese population. JAMA Ophthalmology,134(10), 1087–1093.
Rees, G., Tee, H. W., Marella, M., Fenwick, E., Dirani, M., & Lamoureux, E. L. (2010). Vision-specific distress and depressive symptoms in people with vision impairment. Investigative Ophthalmology & Visual Science,51(6), 2891–2896.
Snyder, C. F., Jensen, R. E., Segal, J. B., & Wu, A. W. (2013). Patient-reported outcomes (PROs): Putting the patient perspective in patient-centered outcomes research. Medical Care,51(8 Suppl 3), S73–S79.
Snyder, C. F., Aaronson, N. K., Choucair, A. K., Elliott, T. E., Greenhalgh, J., Halyard, M. Y., et al. (2012). Implementing patient-reported outcomes assessment in clinical practice: A review of the options and considerations. Quality of Life Research,21(8), 1305–1314.
Greenhalgh, J. (2009). The applications of PROs in clinical practice: What are they, do they work, and why? Quality of Life Research,18(1), 115–123.
Lamoureux, E., Pallant, J. F., Pesudovs, K., Hassell, J. B., & Keeffe, J. E. (2006). The impact of vision impairment questionnaire: An evaluation of its measurement properties using Rasch analysis. Investigative Ophthalmology & Visual Science,47, 4732–4741.
Lamoureux, E., Pallant, J. F., Pesudovs, K., Rees, G., Hassell, J. B., & Keeffe, J. E. (2007). The impact of vision impairment questionnaire: An assessment of its domain structure using confirmatory factor analysis and Rasch analysis. Investigative Ophthalmology & Visual Science,48, 1001–1006.
Fenwick, E. K., Man, R. E., Rees, G., Keeffe, J., Wong, T. Y., & Lamoureux, E. L. (2017). Reducing respondent burden: Validation of the brief impact of vision impairment questionnaire. Quality of Life Research,26(2), 479–488.
Khadka, J., McAlinden, C., & Pesudovs, K. (2013). Quality assessment of ophthalmic questionnaires: Review and recommendations. Optometry and Vision Science,90(8), 720–744.
Gibbons, R. D., Weiss, D. J., Kupfer, D. J., Frank, E., Fagiolini, A., Grochocinski, V. J., et al. (2008). Using computerized adaptive testing to reduce the burden of mental health assessment. Psychiatric Services (Washington, D. C.),59(4), 361–368.
Cella, D., Gershon, R., Lai, J. S., & Choi, S. (2007). The future of outcomes measurement: Item banking, tailored short-forms, and computerized adaptive assessment. Quality of Life Research,16(Suppl), 1133–1141.
Forkmann, T., Boecker, M., Norra, C., Eberle, N., Kircher, T., Schauerte, P., et al. (2009). Development of an item bank for the assessment of depression in persons with mental illnesses and physical diseases using Rasch analysis. Rehabilitation Psychology,54(2), 186–197.
Revicki, D. A., & Cella, D. F. (1997). Health status assessment for the twenty-first century: Item response theory, item banking and computer adaptive testing. Quality of Life Research,6(6), 595–600.
Reeve, B. B., Hays, R. D., Bjorner, J. B., Cook, K. F., Crane, P. K., Teresi, J. A., et al. (2007). Psychometric evaluation and calibration of health-related quality of life item banks: Plans for the patient-reported outcomes measurement information system (PROMIS). Medical Care,45(5 Suppl 1), S22–S31.
Lamoureux, E. L., Pallant, J. F., Pesudovs, K., Rees, G., Hassell, J. B., & Keeffe, J. E. (2007). The effectiveness of low-vision rehabilitation on participation in daily living and quality of life. Investigative Ophthalmology & Visual Science,48(4), 1476–1482.
Goldstein, J., E. Fenwick, R. Finger, V.K. Gothwal, M.L. Jackson, E. Lamoureux, G. Rees, and R.W. Massof. (2018). Calibrating the impact of vision impairment (IVI): Creation of a sample-independent visual function measure for patient-centered outcomes research. Translational Vision Science & Technology, In press
Weih, L. M., Hassell, J. B., & Keeffe, J. (2002). Assessment of the impact of vision impairment. Investigative Ophthalmology & Visual Science,43(4), 927–935.
Rees, G., Xie, J., Chiang, P. P., Larizza, M. F., Marella, M., Hassell, J. B., et al. (2015). A randomised controlled trial of a self-management programme for low vision implemented in low vision rehabilitation services. Patient Education and Counseling,98(2), 174–181.
Rees, G., Xie, J., Holloway, E. E., Sturrock, B. A., Fenwick, E. K., Keeffe, J. E., et al. (2013). Identifying distinct risk factors for vision-specific distress and depressive symptoms in people with vision impairment. Investigative Ophthalmology & Visual Science,54(12), 7431–7438.
Linacre, J. (1994). Sample size and item calibration stability. Rasch Measurement Transactions,7(4), 328.
Linacre, J. M. (2005). A user’s guide to Winsteps/Ministeps Rasch-model programs. Chicago: MESA Press.
Andrich, D. (1978). Rating formulation for ordered response categories. Psychometrica,43, 561–573.
Chen, S. K., & Cook, K. F. (2009). simpolycat: An SAS program for conducting CAT simulation based on polytomous IRT models. Behavior Research Methods,41(2), 499–506.
Choi, S. W. (2009). Firestar: computerized adaptive testing simulation program for polytomous item response theory models. Applied Psychological Measurement,33(8), 644–645.
Latimer, S., Meade, T., & Tennant, A. (2014). Development of item bank to measure deliberate self-harm behaviours: Facilitating tailored scales and computer adaptive testing for specific research and clinical purposes. Psychiatry Research,217(3), 240–247.
Revuelta, J., & Ponsoda, V. (1998). A comparison of item exposure control methods in computerized adaptive testing. Journal of Educational Measurement,35, 311–327.
Embretson, S., & Reise, S. P. (2000). Item response theory for psychologists. New Jersey London: Lawrence Erlbaun Associates.
Scalise, K., & Allen, D. (2015). Use of open-source software for adaptive measurement: Concerto as an R-based computer adaptive development and delivery platform. British Journal of Mathematical and Statistical Psychology,68, 478–496.
Kingsbury, G., & Zara, A. (1989). Procedures for selecting items for computerized adaptive tests. Applied Measurement in Education,2(4), 359–375.
Fenwick, E., Barnard, J., Gan, A., Loe, B. S., Khadka, J., Pesudovs, K., Man, R. E . K., Lee, S. Y., Tan, G., Wong, T., & Lamoureux, E. L. (2019). Computerized adaptive tests: an innovative, efficient, and precise method to assess the patient-centred impact of diabetic retinopathy. Under review.
Gershon, R. C. (2005). Computer adaptive testing. Journal of Applied Measurement,6(1), 109–127.
Haley, S. M., Ni, P., Jette, A. M., Tao, W., Moed, R., Meyers, D., et al. (2009). Replenishing a computerized adaptive test of patient-reported daily activity functioning. Quality of Life Research,18(4), 461–471.
Finger, R., Tellis, B., Crewe, J., Keeffe, J., Ayton, L., & Guymer, R. (2014). Developing the impact of vision impairment-very low vision (IVI-VLV) questionnaire as part of the LoVADA protocol. Investigative Ophthalmology & Visual Science,55(10), 6150–6158.
Bradley, S., Rumsfeld, J., & Ho, P. (2016). Incorporating health status in routine care to improve health care value: The VA patient reported health status assessment (PROST) system. JAMA,316(5), 487–488.
Wu, A.W., R.E. Jensen, C. Salzberg, and C.F. Snyder, Advances in the use of patient reported outcome measures in electronic health records including case studies. 2013, In support of the PCORI National Workshop to Advance the Use of PRO measures in Electronic Health Records: Atlanta, USA.
Lim, A. S., & Bishop, G. D. (2000). The role of attitudes and beliefs in differential health care utilisation among Chinese in Singapore. Psychology and Health,14(6), 965–977.
Fenwick, E., Man, R., Cheung, C., Sabanayagam, C., Cheng, C.-Y., Neelam, K., et al. (2017). Ethnic differences in the association between age-related macular degeneration and vision-specific functioning. JAMA Ophthalmology,135(5), 469–476.
Funding
Prof. Ecosse Lamoureux was supported by an Australian National Health and Medical Research Council (NHMRC) Senior Research Fellowship (#1045280). Dr Gwyn Rees was funded by an NHMRC Career Development Fellowship (#1061801). The funding organizations had no role in the design or conduct of this research or preparation of this manuscript. The Centre for Eye Research Australia receives Operational Infrastructure Support from the Victorian Government.
Author information
Authors and Affiliations
Contributions
EKF contributed to the conception of the work, interpretation of the data, and drafting of the manuscript; EKF and BSL conducted the data analysis, assisted with interpretation of results, and drafted sections of the paper; JK contributed to the methodology and study design and revised the manuscript critically for important intellectual content; GW revised the manuscript critically for important intellectual content; ELL contributed to the conception of the work, interpretation of the data, and revised the manuscript critically for important intellectual content. He will act as a overall guarantor of the study.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee (Royal Victorian Eye and Ear Hospital Human Research Ethics Committee #04/556H, #09/923H) and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed consent
Informed consent was obtained from all individual participants included in the study.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Fenwick, E.K., Loe, B.S., Khadka, J. et al. Optimizing measurement of vision-related quality of life: a computerized adaptive test for the impact of vision impairment questionnaire (IVI-CAT). Qual Life Res 29, 765–774 (2020). https://doi.org/10.1007/s11136-019-02354-y
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11136-019-02354-y