Open Access, Peer-reviewed
pISSN 1015-4817
eISSN 2289-0963
    J Korean Neuropsychiatr Assoc. 2017 Aug;56(3):103-110. Korean.
    Published online Aug 31, 2017.
    https://doi.org/10.4306/jknpa.2017.56.3.103
    Copyright © 2017 Korean Neuropsychiatric Association
    Original Article

    A Case Study of a Machine-Learning Approach in Differential Diagnosis of Schizophrenia: The Predictive Capacity of WAIS-IV

    Eun Hae Ko, MD,1 Hi Yang Kang, PhD,1 Yong Sik Kim, MD, PhD,2 and Seong Hoon Jeong, MD, PhD1
      • 1Department of Psychiatry, Daejeon Eulji Medical Center, Eulji University, Daejeon, Korea.
      • 2Department of Psychiatry, Dongguk University Ilsan Hospital, Goyang, Korea.
    • Address for correspondence: Seong Hoon Jeong, MD, PhD. Department of Psychiatry, Daejeon Eulji Medical Center, Eulji University, 95 Dunsanseo-ro, Seo-gu, Daejeon 35233, Korea. Tel +82-42-611-3443, Fax +82-42-611-3443, Email: AnselmJeong@gmail.com
    Received April 06, 2017; Revised May 11, 2017; Accepted May 26, 2017.

    Abstract

    Objectives

    Machine learning (ML) encompasses a body of statistical approaches that can detect complex interaction patterns from multi-dimensional data. ML is gradually being adopted in medical science, for example, in treatment response prediction and diagnostic classification. Cognitive impairment is a prominent feature of schizophrenia, but is not routinely used in differential diagnosis. In this study, we investigated the predictive capacity of the Wechsler Adult Intelligence Scale IV (WAIS-IV) in differentiating schizophrenia from non-psychotic illnesses using the ML methodology. The purpose of this study was to illustrate the possibility of using ML as an aid in differential diagnosis.

    Methods

    The WAIS-IV test data for 434 psychiatric patients were curated from archived medical records. Using the final diagnoses based on DSM-IV as the target and the WAIS-IV scores as predictor variables, predictive diagnostic models were built using 1) linear 2) non-linear/non-parametric ML algorithms. The accuracy obtained was compared to that of the baseline model built without the WAIS-IV information.

    Results

    The performances of the various ML models were compared. The accuracy of the baseline model was 71.5%, but the best non-linear model showed an accuracy of 84.6%, which was significantly higher than that of non-informative random guessing (p=0.002). Overall, the models using the non-linear algorithms showed better accuracy than the linear ones.

    Conclusion

    The high performance of the developed models demonstrated the predictive capacity of the WAIS-IV and justified the application of ML in psychiatric diagnosis. However, the practical application of ML models may need refinement and larger-scale data collection.

    Keywords
    Machine learning; Schizophrenia; WAIS-IV; Neuropsychological function; Diagnostic support system

    Figures

    Fig. 1
    Receiver operation curve depicting the model performances of the best nonlinear model (radial basis function kernel support vector machine) and the best linear model (penalized logistic regression). RBF-SVM : Radial basis function kernel support vector machine, Lasso : Penalized logistic regression, AUC : Area under the curve.

    Tables

    Table 1
    The final diagnoses of included subjects based on DSM-IV

    Table 2
    Demographic characteristics of the included subjects

    Table 3
    The WAIS-IV subtests, index scores and their comparison between schizophrenia group and non-psychotic illness group

    Table 4
    The performance of the diagnostic models built by different machine learning algorithms

    Notes

    Conflicts of Interest:The authors have no financial conflicts of interest.

    References

      1. Hastie T, Tibshirani R, Friedman J. In: The elements of statistical learning: data mining, inference, and prediction. 2nd ed. New York: Springer Science & Business Media; 2009.
      1. Clarke B, Fokoue E, Zhang HH. In: Principles and theory for data mining and machine learning. New York: Springer Science & Business Media; 2009.
      1. Zhou ZH. In: Ensemble methods: foundations and algorithms. Boca Raton: CRC Press; 2012.
      1. Mikolas P, Melicher T, Skoch A, Matejka M, Slovakova A, Bakstein E, et al. Connectivity of the anterior insula differentiates participants with first-episode schizophrenia spectrum disorders from controls: a machine-learning study. Psychol Med 2016;46:2695–2704.
      1. Deo RC. Machine learning in medicine. Circulation 2015;132:1920–1930.
      1. Ravan M, Hasey G, Reilly JP, MacCrimmon D, Khodayari-Rostamabad A. A machine learning approach using auditory odd-ball responses to investigate the effect of Clozapine therapy. Clin Neurophysiol 2015;126:721–730.
      1. Shim M, Hwang HJ, Kim DW, Lee SH, Im CH. Machine-learning-based diagnosis of schizophrenia using combined sensor-level and source-level EEG features. Schizophr Res 2016;176:314–319.
      1. Johannesen JK, Bi J, Jiang R, Kenney JG, Chen CA. Machine learning identification of EEG features predicting working memory performance in schizophrenia and healthy adults. Neuropsychiatr Electrophysiol 2016;2:3.
      1. Koutsouleris N, Kahn R, Chekroud AM, Leucht S, Falkai P, Wobrock T, et al. Multisite prediction of 4-week and 52-week treatment outcomes in patients with first-episode psychosis: a machine learning approach. Lancet Psychiatry 2016;3:935–946.
      1. Weickert CS, Weickert TW, Pillai A, Buckley PF. Biomarkers in schizophrenia: a brief conceptual consideration. Dis Markers 2013;35:3–9.
      1. Tomasik J, Schwarz E, Guest PC, Bahn S. Blood test for schizophrenia. Eur Arch Psychiatry Clin Neurosci 2012;262 Suppl 2:S79–S83.
      1. Michel NM, Goldberg JO, Heinrichs RW, Miles AA, Ammari N, McDermid Vaz S. WAIS-IV profile of cognition in schizophrenia. Assessment 2013;20:462–473.
      1. Schaefer J, Giangrande E, Weinberger DR, Dickinson D. The global cognitive impairment in schizophrenia: consistent over decades and around the world. Schizophr Res 2013;150:42–50.
      1. Kahn RS, Keefe RS. Schizophrenia is a cognitive illness: time for a change in focus. JAMA Psychiatry 2013;70:1107–1112.
      1. Bora E, Harrison BJ, Yücel M, Pantelis C. Cognitive impairment in euthymic major depressive disorder: a meta-analysis. Psychol Med 2013;43:2017–2026.
      1. Seaton BE, Goldstein G, Allen DN. Sources of heterogeneity in schizophrenia: the role of neuropsychological functioning. Neuropsychol Rev 2001;11:45–67.
      1. Keefe RS, Fenton WS. How should DSM-V criteria for schizophrenia include cognitive impairment? Schizophr Bull 2007;33:912–920.
      1. Dickinson D, Iannone VN, Wilk CM, Gold JM. General and specific cognitive deficits in schizophrenia. Biol Psychiatry 2004;55:826–833.
      1. Dickinson D, Ragland JD, Gold JM, Gur RC. General and specific cognitive deficits in schizophrenia: Goliath defeats David. Biol Psychiatry 2008;64:823–827.
      1. Trivedi MH, Greer TL. Cognitive dysfunction in unipolar depression: implications for treatment. J Affect Disord 2014:152–154. 19–27.
      1. Iwabuchi SJ, Liddle PF, Palaniyappan L. Clinical utility of machine-learning approaches in schizophrenia: improving diagnostic confidence for translational neuroimaging. Front Psychiatry 2013;4:95.
      1. Jordan MI, Mitchell TM. Machine learning: trends, perspectives, and prospects. Science 2015;349:255–260.
      1. Wechsler D. The psychometric tradition: developing the wechsler adult intelligence scale. Contemp Educ Psychol 1981;6:82–85.
      1. Sumiyoshi C, Uetsuki M, Suga M, Kasai K, Sumiyoshi T. Development of brief versions of the Wechsler Intelligence Scale for schizophrenia: considerations of the structure and predictability of intelligence. Psychiatry Res 2013;210:773–779.
      1. Fujino H, Sumiyoshi C, Sumiyoshi T, Yasuda Y, Yamamori H, Ohi K, et al. Performance on the Wechsler Adult Intelligence Scale-III in Japanese patients with schizophrenia. Psychiatry Clin Neurosci 2014;68:534–541.
      1. Heinrichs RW, Zakzanis KK. Neurocognitive deficit in schizophrenia: a quantitative review of the evidence. Neuropsychology 1998;12:426–445.
      1. Hwang ST, Kim JH, Park KB, Choi JY, Hong SH, editors. Standardization of the K-WAIS-IV; Proceedings of Korean Psychological Association Annual Conference; 2012 Aug 24; Chuncheon, Korea. Seoul: Korean Psychological Association; 2012.
      1. McHugh ML. Interrater reliability: the kappa statistic. Biochem Med (Zagreb) 2012;22:276–282.
      1. R Development Core Team. R: a language and environment for statistical computing. 3.2.4 ed. Vienna: R Foundation for Statistical Computing; 2016.
      1. Kuhn M. In: Caret: classification and regression training. 6.0-68 ed. R package version; 2016.
      1. Harrison-Read P. IQ tests as aids to diagnosis and management in early schizophrenia. Adv Psychiatr Treat 2008;14:235–240.
      1. Bora E, Yucel M, Pantelis C. Cognitive functioning in schizophrenia, schizoaffective disorder and affective psychoses: meta-analytic study. Br J Psychiatry 2009;195:475–482.
      1. Keefe RS, Perkins DO, Gu H, Zipursky RB, Christensen BK, Lieberman JA. A longitudinal study of neurocognitive function in individuals at-risk for psychosis. Schizophr Res 2006;88:26–35.
      1. Kitchen H, Rofail D, Heron L, Sacco P. Cognitive impairment associated with schizophrenia: a review of the humanistic burden. Adv Ther 2012;29:148–162.
      1. Dickinson D, Ramsey ME, Gold JM. Overlooking the obvious: a meta-analytic comparison of digit symbol coding tasks and other cognitive measures in schizophrenia. Arch Gen Psychiatry 2007;64:532–542.
      1. Chekroud AM, Zotti RJ, Shehzad Z, Gueorguieva R, Johnson MK, Trivedi MH, et al. Cross-trial prediction of treatment outcome in depression: a machine learning approach. Lancet Psychiatry 2016;3:243–250.
      1. Chekroud AM, Gueorguieva R, Krumholz HM, Trivedi MH, Krystal JH, McCarthy G. Reevaluating the efficacy and predictability of antidepressant treatments: a symptom clustering approach. JAMA Psychiatry 2017;74:370–378.
      1. Manove E, Levy B. Cognitive impairment in bipolar disorder: an overview. Postgrad Med 2010;122:7–16.
      1. Vieta E, Popovic D, Rosa AR, Solé B, Grande I, Frey BN, et al. The clinical implications of cognitive impairment and allostatic load in bipolar disorder. Eur Psychiatry 2013;28:21–29.
      1. Zielesny A. In: From curve fitting to machine learning: an illustrative guide to scientific data analysis and computational intelligence. 2nd ed. Switzerland: Springer International Publishing; 2016.
      1. Han H, Jiang X. Overcome support vector machine diagnosis overfitting. Cancer Inform 2014;13 Suppl 1:145–158.
      1. Holthausen EA, Wiersma D, Sitskoorn MM, Hijman R, Dingemans PM, Schene AH, et al. Schizophrenic patients without neuropsychological deficits: subgroup, disease severity or cognitive compensation? Psychiatry Res 2002;112:1–11.
      1. Allen DN, Goldstein G, Warnick E. A consideration of neuropsychologically normal schizophrenia. J Int Neuropsychol Soc 2003;9:56–63.
      1. Reichenberg A, Harvey PD, Bowie CR, Mojtabai R, Rabinowitz J, Heaton RK, et al. Neuropsychological function and dysfunction in schizophrenia and psychotic affective disorders. Schizophr Bull 2009;35:1022–1029.
    MeSH Terms
    Adult
    Classification
    Cognition Disorders
    Data Collection
    Diagnosis
    Diagnosis, Differential*
    Diagnostic and Statistical Manual of Mental Disorders
    Humans
    Intelligence
    Machine Learning
    Medical Records
    Mental Disorders
    Nonlinear Dynamics
    Schizophrenia*
    Metrics
    Share
    Figures
    slide

    1 / 1

    Tables
    slide
    slide
    slide
    slide

    1 / 4

    PERMALINK