Diabetes mellitus (DM) is a major chronic disease whose prevalence is continuously increasing [1]. Diabetes causes countless issues as it is associated with various complications and eventually plays a role as a major cause (direct or indirect) of death. Therefore, guidelines recommend performing early and regular screening tests for people with diabetes risk factors, including family history of diabetes, prediabetic condition, history of gestational diabetes, and insulin resistance, emphasizing the need for early diagnosis and treatment [2-4]. Furthermore, DM prevention is as important as its management. Once the risk of DM occurrence is recognized, early intervention, including healthy dietary habits and regular physical activity, helps slow down the progression from a prediabetic to a diabetic condition [5-7]. In this context, informing individuals of their risk of diabetes and offering them appropriate lifestyle modifications would be more efficient and effective than treating them once the disease has developed.

Machine learning and data mining models, techniques in the field of artificial intelligence, have been widely used to detect meaningful variables and determine the relationship between them. Recently, these have been introduced in the medical field and used to identify hidden factors and suggest computational patterns based on large databases with various parameters [8-14]. Several models have been developed to predict diabetes under different conditions. However, their application in clinical practice has been limited as they have not provided consistent outcomes across varying datasets, designs, etc.

Many people undergo general routine health checkups provided by the national healthcare system or according to their own needs, and huge datasets of general health checkup findings, available to the patients or for research purposes, have been accumulated over time in Korea [15]. This study aimed to develop valid and applicable DM prediction models using an electronic medical record database as a retrospective cohort study. Implementations of various DM prediction model types were attempted, taking full advantage of the available variables in our database, a dataset of private health checkups conducted at a health promotion center in a tertiary hospital. In Korea, the state requires individuals to undergo routine health examinations. However, the number of health examination variables in the tests provided by the state is much smaller than that in examinations provided by our health promotion center. Therefore, we attempted to create the same DM prediction models with a reduced variable number based on the tests performed during national routine health checkups. The reduced variable number would allow our models to be used with results from not only the relatively intensive private but also simplified national health checkups.

Overall, the medical records contained information of approximately 134,691 individuals of which 3,952 were diagnosed with diabetes. Among these, 752 subjects with data of the previous 24 months were included in DM prediction Model-2, and 641 subjects with data of the previous 12 months were included in DM prediction Model-1. The same number of subjects (752 for Model-1 and 641 for Model-2) were randomly selected from the remaining 130,739 subjects without diabetes and assigned to the non-diabetic groups. Among the diabetic patients, 519 subjects were included in Model-3 after they were found to have prediabetic conditions. The non-diabetic group of Model-3 comprised 519 randomly selected subjects without diabetes whose laboratory results indicated that they were prediabetic 12 months before. We included 281 subjects in Model-4 after limiting those with the data of the previous 24 months (Table 1).

Table 1 shows the performance evaluation parameters of the DM prediction models. All prediction models performed better with 62 variables than with 27 variables, with an AUC difference of over 0.04 between them. Model-2 had the largest AUCs of 0.928 and 0.880 with 62 and 27 variables, respectively. Other respective parameters of this model included accuracy (0.858 and 0.807), recall (0.856 and 0.817), and precision (0.857 and 0.802). For prediabetic subjects, Model-4 had larger AUCs (0.925 and 0.873 with 62 and 27 variables, respectively) than those of Model-3.

Fig. 2 shows the effects of each of the 27 variables on diabetes prediction, listing the variables in decreasing order of importance. Fasting glucose level was the most important prediction variable, followed by body mass index (BMI). The regression coefficient and odds ratio were 0.028 and 1.028, respectively, for fasting glucose levels of –0.123 and 1.013, respectively, for BMI.

The performance of the models changed significantly depending on whether fasting glucose was included. Table 2 shows the importance of fasting glucose in the prediction of diabetes within 2 years. After excluding the fasting glucose level (Model-1A), the accuracy decreased from 0.807 to 0.714, recall decreased from 0.817 to 0.747, precision decreased from 0.802 to 0.702, and AUC decreased from 0. 878 to 0.793. Models 1B, 1C, and 1D, which included fasting glucose, had performance parameter values similar to those of Model-1. When building a model with fasting blood glucose alone, the accuracy was 0.803 (Model-1B). Model-1E, which included the two most powerful variables, fasting blood glucose and HbA1c, had an accuracy of 0.820, a recall of 0.826, a precision of 0.818, and an AUC of 0.898. Supplementary Fig. 2 shows the distribution of the study population according to fasting glucose levels. The distribution graph of the diabetic group seems to have shifted to the right, compared to that of the non-diabetic group. The mean and standard deviation values were 107.27±9.90 mg/dL for the diabetic group and 91.36±9.79 mg/dL for the non-diabetic group. Fasting blood glucose with a cutoff value of 99 mg/dL is expected to predict the presence of diabetes within 2 years with an accuracy of 0.803.

In this study, we developed 1- and 2-year diabetes prediction models using electronic medical records from general health checkups at Seoul St. Mary’s Hospital. All models showed good performance with AUCs of approximately 0.9. Silva et al. [17] suggested an AUC of 0.812 for machine learning prediction models in their systematic review and meta-analysis. In comparison, the results of this study are satisfactory.

There are several possible reasons for the outstanding performance of our predictive models despite using a relatively small sample size. We used diverse variables including self-reported information, physical examinations, and laboratory tests. A previous study with only the information from a questionnaire showed an AUC of 0.766 despite using a larger sample size of 18,301 subjects [18]. Our models with 62 variables included pulmonary function test results. It was suggested that some of them were independent risk factors for developing diabetes, where forced vital capacity and forced expiratory volume in one-second correlated significantly with the incidence of diabetes [19]. However, to the best of our knowledge, this is the first study to present a machine learning diabetes predictive model, particularly using pulmonary function test results. Unfortunately, some meaningful variables, such as high-sensitivity C-reactive protein and adiponectin, were not included because we obtained less than 20,000 measurements/data points for these. Other meaningful variables, such as stress, occupation, and education, were not included as they were not routinely checked or recorded [20]. We also did not include information from imaging studies as they cannot be automatically entered into the computed dataset because standardized description, definite categorization, and data processing were not yet established.

We included fasting glucose and HbA1c levels in our model. We decided to include them because of their clinical importance. The importance of the two parameters was well proven by evaluating the performance of the 2-year prediction model with and without these variables. Our results are consistent with those of a Japanese study in which the AUC increased from 0.717 to 0.893 and from 0.734 to 0.882 by adding fasting glucose and HbA1c levels, respectively [21]. Other studies also showed similar degrees of improvement when fasting glucose and HbA1c levels were included [22-27].

The prediction time was limited to 1 or 2 years. A previous study developed a prediction model with a lower AUC of 0.820 after analyzing 93 parameters of 3,363 individuals; however, they made an eight-year prediction [20]. Generally, a longer prediction time is known to be more difficult. The performances of the 1-year prediction were well maintained with a similarly small sample size when extended to the 2-year prediction in our study. Our 2-year prediction model had a performance accuracy of over 80% when fasting blood glucose levels exceeded a certain threshold. Furthermore, the diabetic and non-diabetic groups had distinct distributions according to fasting blood glucose levels (Supplementary Fig. 2). In this context, our models could have an even better performance in predicting diabetes over a longer period such as 5 years.

One of the study strengths was data management [28,29]. We decided to perform simple median imputation and hyperparameter tuning because little had changed in the performance with the exponential smoothing model, last observation carried forward method, Holt-Winters method, and more.

The study population was limited to those who underwent general checkups in one center, sharing common characteristics such as homogeneous Korean ethnicity and similar socioeconomic statuses. Diabetes pathophysiology differs between ethnic groups and socioeconomic environments. For instance, the Framingham Diabetes Risk Scoring model [30], a well-known diabetes predictor based on a middle-aged United States population with an AUC of 0.850, showed different performance when applied to a Canadian population, with an AUC of 0.78 [31].

Our goal was to reduce the economic burden on society by preventing diabetes rather than treating it. We developed models that deliver outstanding performance using 62 easily obtainable and commonly measured variables. Simplified models with 27 variables present in the national health checkups were inferior to our models with 62 variables, but their performance parameters were still similar or better than those in previous reports. As the Korean National Health Insurance Service (NHIS) provides for medical checkups every 2 years for all Koreans [15], our models could be readily applied to anyone who underwent the simple free national checkups. Individuals could recognize their risk of developing diabetes in the near future, within 1 or 2 years in this study, and individualized medical advice could be applied to prevent diabetes. Facing the specified risk could motivate people at risk to take action and emphasize a healthy lifestyle, especially for those at high-risk of diabetes. Physicians could encourage strong intervention and short-term follow-up, and politicians could devise strategies for a stratified approach to slow down the progress of DM (a society program). In this study, the model with the two most powerful variables, fasting blood glucose and HbA1c, outperformed the model with 27 variables included in NHIS checkups. Rhee et al. [32] developed a diabetes prediction model using the NHIS cohort of Korea, which outperformed the conventional model. If they can add HbA1c to their model, they might achieve outstanding performance. Therefore, we could ask the national health policymakers to add HbA1c in the routine examination for nationwide general checkups after making cost-effectiveness calculations. This will help prevent high-risk individuals from proceeding to develop diabetes as well as find undiagnosed diabetic patients, so that they can be treated. It should be worthwhile because the model with only fasting plasma glucose and HbA1c outperformed the model with the 27 national general checkup variables. We hope to determine the effectiveness of detection, followed by intervention. Fasting glucose level is important in both the diagnosis and prediction of diabetes. Kim et al. [33] suggested that higher fasting glucose levels, even within the normal range, lead to a risk of diabetes. We expect to predict the risk of diabetes in populations with normal fasting glucose levels in another study.

This study has some limitations. Our dataset was obtained from a tertiary institution. Typically, patients with more serious and multiple diseases tend to visit higher level hospitals. However, this tendency is not expected to be too high as the data were from a screening service for diseases as general checkups and not an examination of patients, and we included a large number of examinees, especially those who underwent at least two examinations approximately 2 years apart. The examinees visited according to their own desire to be screened. They were not selected by searching for the international classification of disease codes. The model was validated internally and not externally. As we have no information except that of 62 variables, we cannot confirm that our study subjects are representative of the general population. Therefore, it would need to be validated before being applied to other situations. Typically, machine learning prediction models require validation when conditions change. It is preferable to create a specific prediction model for each location and regularly check its performance. Non-diabetic groups were randomly selected from those who met our conditions. We neither focused on the characteristics of each group nor used matching techniques to specify the weights of importance of specific variables. Instead, we put all variables together, including age and sex, and evaluated the effects of all these as real-world data.

In conclusion, we developed a high-performance model to predict diabetes within 1 and 2 years. We expect the risk predictions suggested by the model to help health providers and clinicians provide more specific advice and encourage the examinees to lead a healthy lifestyle. We hope that this study will become a foundation for further clinical trials to elucidate the application of diabetes risk calculators, early interventions, and their diabetes prevention effects.