A Case Study of a Machine-Learning Approach in Differential Diagnosis of Schizophrenia: The Predictive Capacity of WAIS-IV

Eun Hae Ko; Hi Yang Kang; Yong Sik Kim; Seong Hoon Jeong

doi:10.4306/jknpa.2017.56.3.103

Journal List > J Korean Neuropsychiatr Assoc > v.56(3) > 1017841

Go to TopGo to Top Go to BottomGo to Bottom

TOOLS

Ko, Kang, Kim, and Jeong: A Case Study of a Machine-Learning Approach in Differential Diagnosis of Schizophrenia: The Predictive Capacity of WAIS-IV

Original Article

Journal of Korean Neuropsychiatric Association 2017; 56(3): 103-110.

Published online: 31 August 2017

DOI: https://doi.org/10.4306/jknpa.2017.56.3.103

A Case Study of a Machine-Learning Approach in Differential Diagnosis of Schizophrenia: The Predictive Capacity of WAIS-IV

Eun Hae Ko, MD¹, Hi Yang Kang, PhD¹, Yong Sik Kim, MD, PhD², Seong Hoon Jeong, MD, PhD¹

¹Department of Psychiatry, Daejeon Eulji Medical Center, Eulji University, Daejeon, Korea.

²Department of Psychiatry, Dongguk University Ilsan Hospital, Goyang, Korea.

Address for correspondence: Seong Hoon Jeong, MD, PhD. Department of Psychiatry, Daejeon Eulji Medical Center, Eulji University, 95 Dunsanseo-ro, Seo-gu, Daejeon 35233, Korea. Tel +82-42-611-3443, Fax +82-42-611-3443, AnselmJeong@gmail.com

Received 6 April 2017 Revised 11 May 2017 Accepted 26 May 2017

Abstract

Objectives

Machine learning (ML) encompasses a body of statistical approaches that can detect complex interaction patterns from multi-dimensional data. ML is gradually being adopted in medical science, for example, in treatment response prediction and diagnostic classification. Cognitive impairment is a prominent feature of schizophrenia, but is not routinely used in differential diagnosis. In this study, we investigated the predictive capacity of the Wechsler Adult Intelligence Scale IV (WAIS-IV) in differentiating schizophrenia from non-psychotic illnesses using the ML methodology. The purpose of this study was to illustrate the possibility of using ML as an aid in differential diagnosis.

Methods

The WAIS-IV test data for 434 psychiatric patients were curated from archived medical records. Using the final diagnoses based on DSM-IV as the target and the WAIS-IV scores as predictor variables, predictive diagnostic models were built using 1) linear 2) non-linear/non-parametric ML algorithms. The accuracy obtained was compared to that of the baseline model built without the WAIS-IV information.

Results

The performances of the various ML models were compared. The accuracy of the baseline model was 71.5%, but the best non-linear model showed an accuracy of 84.6%, which was significantly higher than that of non-informative random guessing (p=0.002). Overall, the models using the non-linear algorithms showed better accuracy than the linear ones.

Conclusion

The high performance of the developed models demonstrated the predictive capacity of the WAIS-IV and justified the application of ML in psychiatric diagnosis. However, the practical application of ML models may need refinement and larger-scale data collection.

Keywords: Machine learning, Schizophrenia, WAIS-IV, Neuropsychological function, Diagnostic support system

Figures and Tables

Fig. 1

Receiver operation curve depicting the model performances of the best nonlinear model (radial basis function kernel support vector machine) and the best linear model (penalized logistic regression). RBF-SVM : Radial basis function kernel support vector machine, Lasso : Penalized logistic regression, AUC : Area under the curve.

Table 1

The final diagnoses of included subjects based on DSM-IV

DSM : Diagnostic and Statistical Manual of Mental Disorder

Table 2

Demographic characteristics of the included subjects

^* : Mean±standard deviation

Table 3

The WAIS-IV subtests, index scores and their comparison between schizophrenia group and non-psychotic illness group

^* : Mean±standard deviation. WAIS-IV : Wechsler Adult Intelligence Scale fourth edition, IQ : Intelligence quotient

Table 4

The performance of the diagnostic models built by different machine learning algorithms

SVM : Support vector machine, LDA : Linear discriminant analysis

Notes

^{Conflicts of Interest} The authors have no financial conflicts of interest.

References

1. Hastie T, Tibshirani R, Friedman J. The elements of statistical learning: data mining, inference, and prediction. 2nd ed. New York: Springer Science & Business Media;2009.

2. Clarke B, Fokoue E, Zhang HH. Principles and theory for data mining and machine learning. New York: Springer Science & Business Media;2009.

3. Zhou ZH. Ensemble methods: foundations and algorithms. Boca Raton: CRC Press;2012.

4. Mikolas P, Melicher T, Skoch A, Matejka M, Slovakova A, Bakstein E, et al. Connectivity of the anterior insula differentiates participants with first-episode schizophrenia spectrum disorders from controls: a machine-learning study. Psychol Med. 2016; 46:2695–2704.

5. Deo RC. Machine learning in medicine. Circulation. 2015; 132:1920–1930.

6. Ravan M, Hasey G, Reilly JP, MacCrimmon D, Khodayari-Rostamabad A. A machine learning approach using auditory odd-ball responses to investigate the effect of Clozapine therapy. Clin Neurophysiol. 2015; 126:721–730.

7. Shim M, Hwang HJ, Kim DW, Lee SH, Im CH. Machine-learning-based diagnosis of schizophrenia using combined sensor-level and source-level EEG features. Schizophr Res. 2016; 176:314–319.

8. Johannesen JK, Bi J, Jiang R, Kenney JG, Chen CA. Machine learning identification of EEG features predicting working memory performance in schizophrenia and healthy adults. Neuropsychiatr Electrophysiol. 2016; 2:3.

9. Koutsouleris N, Kahn R, Chekroud AM, Leucht S, Falkai P, Wobrock T, et al. Multisite prediction of 4-week and 52-week treatment outcomes in patients with first-episode psychosis: a machine learning approach. Lancet Psychiatry. 2016; 3:935–946.

10. Weickert CS, Weickert TW, Pillai A, Buckley PF. Biomarkers in schizophrenia: a brief conceptual consideration. Dis Markers. 2013; 35:3–9.

11. Tomasik J, Schwarz E, Guest PC, Bahn S. Blood test for schizophrenia. Eur Arch Psychiatry Clin Neurosci. 2012; 262:Suppl 2. S79–S83.

12. Michel NM, Goldberg JO, Heinrichs RW, Miles AA, Ammari N, McDermid Vaz S. WAIS-IV profile of cognition in schizophrenia. Assessment. 2013; 20:462–473.

13. Schaefer J, Giangrande E, Weinberger DR, Dickinson D. The global cognitive impairment in schizophrenia: consistent over decades and around the world. Schizophr Res. 2013; 150:42–50.

14. Kahn RS, Keefe RS. Schizophrenia is a cognitive illness: time for a change in focus. JAMA Psychiatry. 2013; 70:1107–1112.

15. Bora E, Harrison BJ, Yücel M, Pantelis C. Cognitive impairment in euthymic major depressive disorder: a meta-analysis. Psychol Med. 2013; 43:2017–2026.

16. Seaton BE, Goldstein G, Allen DN. Sources of heterogeneity in schizophrenia: the role of neuropsychological functioning. Neuropsychol Rev. 2001; 11:45–67.

17. Keefe RS, Fenton WS. How should DSM-V criteria for schizophrenia include cognitive impairment? Schizophr Bull. 2007; 33:912–920.

18. Dickinson D, Iannone VN, Wilk CM, Gold JM. General and specific cognitive deficits in schizophrenia. Biol Psychiatry. 2004; 55:826–833.

19. Dickinson D, Ragland JD, Gold JM, Gur RC. General and specific cognitive deficits in schizophrenia: Goliath defeats David. Biol Psychiatry. 2008; 64:823–827.

20. Trivedi MH, Greer TL. Cognitive dysfunction in unipolar depression: implications for treatment. J Affect Disord. 2014; 152–154. 19–27.

21. Iwabuchi SJ, Liddle PF, Palaniyappan L. Clinical utility of machine-learning approaches in schizophrenia: improving diagnostic confidence for translational neuroimaging. Front Psychiatry. 2013; 4:95.

22. Jordan MI, Mitchell TM. Machine learning: trends, perspectives, and prospects. Science. 2015; 349:255–260.

23. Wechsler D. The psychometric tradition: developing the wechsler adult intelligence scale. Contemp Educ Psychol. 1981; 6:82–85.

24. Sumiyoshi C, Uetsuki M, Suga M, Kasai K, Sumiyoshi T. Development of brief versions of the Wechsler Intelligence Scale for schizophrenia: considerations of the structure and predictability of intelligence. Psychiatry Res. 2013; 210:773–779.

25. Fujino H, Sumiyoshi C, Sumiyoshi T, Yasuda Y, Yamamori H, Ohi K, et al. Performance on the Wechsler Adult Intelligence Scale-III in Japanese patients with schizophrenia. Psychiatry Clin Neurosci. 2014; 68:534–541.

26. Heinrichs RW, Zakzanis KK. Neurocognitive deficit in schizophrenia: a quantitative review of the evidence. Neuropsychology. 1998; 12:426–445.

27. In : Hwang ST, Kim JH, Park KB, Choi JY, Hong SH, editors. Standardization of the K-WAIS-IV. Proceedings of Korean Psychological Association Annual Conference; 2012 Aug 24; Chuncheon, Korea. Seoul: Korean Psychological Association;2012.

28. McHugh ML. Interrater reliability: the kappa statistic. Biochem Med (Zagreb). 2012; 22:276–282.

29. R Development Core Team. R: a language and environment for statistical computing. 3.2.4 ed. Vienna: R Foundation for Statistical Computing;2016.

30. Kuhn M. Caret: classification and regression training. 6.0-68 ed. R package version;2016.

31. Harrison-Read P. IQ tests as aids to diagnosis and management in early schizophrenia. Adv Psychiatr Treat. 2008; 14:235–240.

32. Bora E, Yucel M, Pantelis C. Cognitive functioning in schizophrenia, schizoaffective disorder and affective psychoses: meta-analytic study. Br J Psychiatry. 2009; 195:475–482.

33. Keefe RS, Perkins DO, Gu H, Zipursky RB, Christensen BK, Lieberman JA. A longitudinal study of neurocognitive function in individuals at-risk for psychosis. Schizophr Res. 2006; 88:26–35.

34. Kitchen H, Rofail D, Heron L, Sacco P. Cognitive impairment associated with schizophrenia: a review of the humanistic burden. Adv Ther. 2012; 29:148–162.

35. Dickinson D, Ramsey ME, Gold JM. Overlooking the obvious: a meta-analytic comparison of digit symbol coding tasks and other cognitive measures in schizophrenia. Arch Gen Psychiatry. 2007; 64:532–542.

36. Chekroud AM, Zotti RJ, Shehzad Z, Gueorguieva R, Johnson MK, Trivedi MH, et al. Cross-trial prediction of treatment outcome in depression: a machine learning approach. Lancet Psychiatry. 2016; 3:243–250.

37. Chekroud AM, Gueorguieva R, Krumholz HM, Trivedi MH, Krystal JH, McCarthy G. Reevaluating the efficacy and predictability of antidepressant treatments: a symptom clustering approach. JAMA Psychiatry. 2017; 74:370–378.

38. Manove E, Levy B. Cognitive impairment in bipolar disorder: an overview. Postgrad Med. 2010; 122:7–16.

39. Vieta E, Popovic D, Rosa AR, Solé B, Grande I, Frey BN, et al. The clinical implications of cognitive impairment and allostatic load in bipolar disorder. Eur Psychiatry. 2013; 28:21–29.

40. Zielesny A. From curve fitting to machine learning: an illustrative guide to scientific data analysis and computational intelligence. 2nd ed. Switzerland: Springer International Publishing;2016.

41. Han H, Jiang X. Overcome support vector machine diagnosis overfitting. Cancer Inform. 2014; 13:Suppl 1. 145–158.

42. Holthausen EA, Wiersma D, Sitskoorn MM, Hijman R, Dingemans PM, Schene AH, et al. Schizophrenic patients without neuropsychological deficits: subgroup, disease severity or cognitive compensation? Psychiatry Res. 2002; 112:1–11.

43. Allen DN, Goldstein G, Warnick E. A consideration of neuropsychologically normal schizophrenia. J Int Neuropsychol Soc. 2003; 9:56–63.

44. Reichenberg A, Harvey PD, Bowie CR, Mojtabai R, Rabinowitz J, Heaton RK, et al. Neuropsychological function and dysfunction in schizophrenia and psychotic affective disorders. Schizophr Bull. 2009; 35:1022–1029.

TOOLS

Similar articles