Journal List > J Korean Neuropsychiatr Assoc > v.56(3) > 1017841

Ko, Kang, Kim, and Jeong: A Case Study of a Machine-Learning Approach in Differential Diagnosis of Schizophrenia: The Predictive Capacity of WAIS-IV

Abstract

Objectives

Machine learning (ML) encompasses a body of statistical approaches that can detect complex interaction patterns from multi-dimensional data. ML is gradually being adopted in medical science, for example, in treatment response prediction and diagnostic classification. Cognitive impairment is a prominent feature of schizophrenia, but is not routinely used in differential diagnosis. In this study, we investigated the predictive capacity of the Wechsler Adult Intelligence Scale IV (WAIS-IV) in differentiating schizophrenia from non-psychotic illnesses using the ML methodology. The purpose of this study was to illustrate the possibility of using ML as an aid in differential diagnosis.

Methods

The WAIS-IV test data for 434 psychiatric patients were curated from archived medical records. Using the final diagnoses based on DSM-IV as the target and the WAIS-IV scores as predictor variables, predictive diagnostic models were built using 1) linear 2) non-linear/non-parametric ML algorithms. The accuracy obtained was compared to that of the baseline model built without the WAIS-IV information.

Results

The performances of the various ML models were compared. The accuracy of the baseline model was 71.5%, but the best non-linear model showed an accuracy of 84.6%, which was significantly higher than that of non-informative random guessing (p=0.002). Overall, the models using the non-linear algorithms showed better accuracy than the linear ones.

Conclusion

The high performance of the developed models demonstrated the predictive capacity of the WAIS-IV and justified the application of ML in psychiatric diagnosis. However, the practical application of ML models may need refinement and larger-scale data collection.

Figures and Tables

Fig. 1

Receiver operation curve depicting the model performances of the best nonlinear model (radial basis function kernel support vector machine) and the best linear model (penalized logistic regression). RBF-SVM : Radial basis function kernel support vector machine, Lasso : Penalized logistic regression, AUC : Area under the curve.

jkna-56-103-g001
Table 1

The final diagnoses of included subjects based on DSM-IV

jkna-56-103-i001

DSM : Diagnostic and Statistical Manual of Mental Disorder

Table 2

Demographic characteristics of the included subjects

jkna-56-103-i002

* : Mean±standard deviation

Table 3

The WAIS-IV subtests, index scores and their comparison between schizophrenia group and non-psychotic illness group

jkna-56-103-i003

* : Mean±standard deviation. WAIS-IV : Wechsler Adult Intelligence Scale fourth edition, IQ : Intelligence quotient

Table 4

The performance of the diagnostic models built by different machine learning algorithms

jkna-56-103-i004

SVM : Support vector machine, LDA : Linear discriminant analysis

Notes

Conflicts of Interest The authors have no financial conflicts of interest.

References

1. Hastie T, Tibshirani R, Friedman J. The elements of statistical learning: data mining, inference, and prediction. 2nd ed. New York: Springer Science & Business Media;2009.
2. Clarke B, Fokoue E, Zhang HH. Principles and theory for data mining and machine learning. New York: Springer Science & Business Media;2009.
3. Zhou ZH. Ensemble methods: foundations and algorithms. Boca Raton: CRC Press;2012.
4. Mikolas P, Melicher T, Skoch A, Matejka M, Slovakova A, Bakstein E, et al. Connectivity of the anterior insula differentiates participants with first-episode schizophrenia spectrum disorders from controls: a machine-learning study. Psychol Med. 2016; 46:2695–2704.
crossref
5. Deo RC. Machine learning in medicine. Circulation. 2015; 132:1920–1930.
crossref
6. Ravan M, Hasey G, Reilly JP, MacCrimmon D, Khodayari-Rostamabad A. A machine learning approach using auditory odd-ball responses to investigate the effect of Clozapine therapy. Clin Neurophysiol. 2015; 126:721–730.
crossref
7. Shim M, Hwang HJ, Kim DW, Lee SH, Im CH. Machine-learning-based diagnosis of schizophrenia using combined sensor-level and source-level EEG features. Schizophr Res. 2016; 176:314–319.
crossref
8. Johannesen JK, Bi J, Jiang R, Kenney JG, Chen CA. Machine learning identification of EEG features predicting working memory performance in schizophrenia and healthy adults. Neuropsychiatr Electrophysiol. 2016; 2:3.
crossref
9. Koutsouleris N, Kahn R, Chekroud AM, Leucht S, Falkai P, Wobrock T, et al. Multisite prediction of 4-week and 52-week treatment outcomes in patients with first-episode psychosis: a machine learning approach. Lancet Psychiatry. 2016; 3:935–946.
crossref
10. Weickert CS, Weickert TW, Pillai A, Buckley PF. Biomarkers in schizophrenia: a brief conceptual consideration. Dis Markers. 2013; 35:3–9.
crossref
11. Tomasik J, Schwarz E, Guest PC, Bahn S. Blood test for schizophrenia. Eur Arch Psychiatry Clin Neurosci. 2012; 262:Suppl 2. S79–S83.
crossref
12. Michel NM, Goldberg JO, Heinrichs RW, Miles AA, Ammari N, McDermid Vaz S. WAIS-IV profile of cognition in schizophrenia. Assessment. 2013; 20:462–473.
crossref
13. Schaefer J, Giangrande E, Weinberger DR, Dickinson D. The global cognitive impairment in schizophrenia: consistent over decades and around the world. Schizophr Res. 2013; 150:42–50.
crossref
14. Kahn RS, Keefe RS. Schizophrenia is a cognitive illness: time for a change in focus. JAMA Psychiatry. 2013; 70:1107–1112.
15. Bora E, Harrison BJ, Yücel M, Pantelis C. Cognitive impairment in euthymic major depressive disorder: a meta-analysis. Psychol Med. 2013; 43:2017–2026.
crossref
16. Seaton BE, Goldstein G, Allen DN. Sources of heterogeneity in schizophrenia: the role of neuropsychological functioning. Neuropsychol Rev. 2001; 11:45–67.
17. Keefe RS, Fenton WS. How should DSM-V criteria for schizophrenia include cognitive impairment? Schizophr Bull. 2007; 33:912–920.
crossref
18. Dickinson D, Iannone VN, Wilk CM, Gold JM. General and specific cognitive deficits in schizophrenia. Biol Psychiatry. 2004; 55:826–833.
crossref
19. Dickinson D, Ragland JD, Gold JM, Gur RC. General and specific cognitive deficits in schizophrenia: Goliath defeats David. Biol Psychiatry. 2008; 64:823–827.
crossref
20. Trivedi MH, Greer TL. Cognitive dysfunction in unipolar depression: implications for treatment. J Affect Disord. 2014; 152–154. 19–27.
crossref
21. Iwabuchi SJ, Liddle PF, Palaniyappan L. Clinical utility of machine-learning approaches in schizophrenia: improving diagnostic confidence for translational neuroimaging. Front Psychiatry. 2013; 4:95.
crossref
22. Jordan MI, Mitchell TM. Machine learning: trends, perspectives, and prospects. Science. 2015; 349:255–260.
crossref
23. Wechsler D. The psychometric tradition: developing the wechsler adult intelligence scale. Contemp Educ Psychol. 1981; 6:82–85.
crossref
24. Sumiyoshi C, Uetsuki M, Suga M, Kasai K, Sumiyoshi T. Development of brief versions of the Wechsler Intelligence Scale for schizophrenia: considerations of the structure and predictability of intelligence. Psychiatry Res. 2013; 210:773–779.
crossref
25. Fujino H, Sumiyoshi C, Sumiyoshi T, Yasuda Y, Yamamori H, Ohi K, et al. Performance on the Wechsler Adult Intelligence Scale-III in Japanese patients with schizophrenia. Psychiatry Clin Neurosci. 2014; 68:534–541.
crossref
26. Heinrichs RW, Zakzanis KK. Neurocognitive deficit in schizophrenia: a quantitative review of the evidence. Neuropsychology. 1998; 12:426–445.
crossref
27. In : Hwang ST, Kim JH, Park KB, Choi JY, Hong SH, editors. Standardization of the K-WAIS-IV. Proceedings of Korean Psychological Association Annual Conference; 2012 Aug 24; Chuncheon, Korea. Seoul: Korean Psychological Association;2012.
28. McHugh ML. Interrater reliability: the kappa statistic. Biochem Med (Zagreb). 2012; 22:276–282.
crossref
29. R Development Core Team. R: a language and environment for statistical computing. 3.2.4 ed. Vienna: R Foundation for Statistical Computing;2016.
30. Kuhn M. Caret: classification and regression training. 6.0-68 ed. R package version;2016.
31. Harrison-Read P. IQ tests as aids to diagnosis and management in early schizophrenia. Adv Psychiatr Treat. 2008; 14:235–240.
32. Bora E, Yucel M, Pantelis C. Cognitive functioning in schizophrenia, schizoaffective disorder and affective psychoses: meta-analytic study. Br J Psychiatry. 2009; 195:475–482.
crossref
33. Keefe RS, Perkins DO, Gu H, Zipursky RB, Christensen BK, Lieberman JA. A longitudinal study of neurocognitive function in individuals at-risk for psychosis. Schizophr Res. 2006; 88:26–35.
crossref
34. Kitchen H, Rofail D, Heron L, Sacco P. Cognitive impairment associated with schizophrenia: a review of the humanistic burden. Adv Ther. 2012; 29:148–162.
crossref
35. Dickinson D, Ramsey ME, Gold JM. Overlooking the obvious: a meta-analytic comparison of digit symbol coding tasks and other cognitive measures in schizophrenia. Arch Gen Psychiatry. 2007; 64:532–542.
36. Chekroud AM, Zotti RJ, Shehzad Z, Gueorguieva R, Johnson MK, Trivedi MH, et al. Cross-trial prediction of treatment outcome in depression: a machine learning approach. Lancet Psychiatry. 2016; 3:243–250.
crossref
37. Chekroud AM, Gueorguieva R, Krumholz HM, Trivedi MH, Krystal JH, McCarthy G. Reevaluating the efficacy and predictability of antidepressant treatments: a symptom clustering approach. JAMA Psychiatry. 2017; 74:370–378.
crossref
38. Manove E, Levy B. Cognitive impairment in bipolar disorder: an overview. Postgrad Med. 2010; 122:7–16.
crossref
39. Vieta E, Popovic D, Rosa AR, Solé B, Grande I, Frey BN, et al. The clinical implications of cognitive impairment and allostatic load in bipolar disorder. Eur Psychiatry. 2013; 28:21–29.
crossref
40. Zielesny A. From curve fitting to machine learning: an illustrative guide to scientific data analysis and computational intelligence. 2nd ed. Switzerland: Springer International Publishing;2016.
41. Han H, Jiang X. Overcome support vector machine diagnosis overfitting. Cancer Inform. 2014; 13:Suppl 1. 145–158.
crossref
42. Holthausen EA, Wiersma D, Sitskoorn MM, Hijman R, Dingemans PM, Schene AH, et al. Schizophrenic patients without neuropsychological deficits: subgroup, disease severity or cognitive compensation? Psychiatry Res. 2002; 112:1–11.
crossref
43. Allen DN, Goldstein G, Warnick E. A consideration of neuropsychologically normal schizophrenia. J Int Neuropsychol Soc. 2003; 9:56–63.
crossref
44. Reichenberg A, Harvey PD, Bowie CR, Mojtabai R, Rabinowitz J, Heaton RK, et al. Neuropsychological function and dysfunction in schizophrenia and psychotic affective disorders. Schizophr Bull. 2009; 35:1022–1029.
crossref
TOOLS
Similar articles