This article has been
cited by other articles in ScienceCentral.
Abstract
Objectives
This study aimed to develop and validate a model to predict the need for intensive care unit (ICU) admission in patients with dental infections using an automated machine learning (ML) program called H2O-AutoML.
Materials and Methods
Two models were created using only the information available at the initial examination. Model 1 was parameterized with only clinical symptoms and blood tests, excluding contrast-enhanced multi-detector computed tomography (MDCT) images available at the initial visit, whereas model 2 was created with the addition of the MDCT information to the model 1 parameters. Although model 2 was expected to be superior to model 1, we wanted to independently determine this conclusion. A total of 210 patients who visited the Department of Oral and Maxillofacial Surgery at the Dankook University Dental Hospital from March 2013 to August 2023 was included in this study. The patients’ demographic characteristics (sex, age, and place of residence), systemic factors (hypertension, diabetes mellitus [DM], kidney disease, liver disease, heart disease, anticoagulation therapy, and osteoporosis), local factors (smoking status, site of infection, postoperative wound infection, dysphagia, odynophagia, and trismus), and factors known from initial blood tests were obtained from their medical charts and retrospectively reviewed.
Results
The generalized linear model algorithm provided the best diagnostic accuracy, with an area under the receiver operating characteristic values of 0.8289 in model 1 and 0.8415 in model 2. In both models, the C-reactive protein level was the most important variable, followed by DM.
Conclusion
This study provides unprecedented data on the use of ML for successful prediction of ICU admission based on initial examination results. These findings will considerably contribute to the development of the field of dentistry, especially oral and maxillofacial surgery.
Keywords: Intensive care units, Infections, C-reactive protein, Machine learning, Algorithms
I. Introduction
Recently, the medical sector has been at the forefront of development of machine learning (ML) models designed to assist doctors in making decisions related to the diagnosis and treatment of complex diseases with multiple factors
1-3. ML is a field of artificial intelligence focused on the creation and analysis of statistical algorithms. It generalizes and executes tasks proficiently without requiring explicit instructions. Simply put, it is a program that educates algorithms using pre-established input-output relationships in data, allowing them to reliably predict output data for new input information
4,5. With the recent increase in the demand for ML, many automated ML tools are available for use by beginners.
Dental infections can cause cellulitis, abscesses, and fasciitis, potentially requiring intensive care unit (ICU) admission depending on their severity. Generally, the decision to admit a patient to the ICU is based on disease severity, whether the surgery was performed under general anesthesia, and the patient’s postoperative condition. When determining disease severity, blood test results and enhanced multi-detector computed tomography (MDCT) images are used. However, the subjective opinion of the operator also is involved in the ICU admission decision process. Furthermore, contrast MDCT imaging requires a 4-hour fast and may be difficult to perform if there is a contrast allergy or chronic kidney disease. In this study, we created a model with demographic, systemic, and local factors as parameters using only the information available at the initial consultation, without considering MDCT images, and focused on how it differed from the model with MDCT images.
The purpose of this study was to develop a ML model that can predict the need for ICU admission using a program called H2O-AutoML in patients with dental infections and to create a model using only the information available at the initial consultation. We created two models: model 1, which was parameterized with only clinical symptoms and blood tests available at the initial consultation, except for enhanced MDCT images, and model 2, which was created by adding information from MDCT images to model 1 parameters. The first aim is to develop a ML-based prediction model, while the second aim is to compare models 1 and 2 depending on inclusion of MDCT imaging.
II. Materials and Methods
1. Patients
This study included patients who visited the Department of Oral and Maxillofacial Surgery, College of Dentistry, Dankook University, from March 2013 to August 2023. Medical and dental history questionnaires were administered to the patients, and their initial examination records and blood laboratory tests were retrospectively reviewed. Patients with incomplete initial examination records or blood laboratory test results were excluded.
A total of 210 patients was included in the study. We considered their demographics (sex, age, and place of residence), systemic variables (hypertension, diabetes mellitus [DM], kidney disease, liver disease, heart disease, anticoagulant use, and osteoporosis), and local factors (smoking status, site of infection, postoperative wound infection, odynophagia, dysphagia, and trismus). We also included factors that could be identified in the initial blood test such as creatinine, blood urea nitrogen, C-reactive protein (CRP), hemoglobin, neutrophil, lymphocyte, monocyte, eosinophil, red blood cell, hematocrit, platelet, glucose, sodium, potassium, and chloride levels. As an additional factor for model 2, we included information about the infected fascial spaces such as the canine, buccal, infratemporal, submental, sublingual, submandibular, masseteric, pterygomandibular, temporal, and deep neck spaces. The output was dichotomized according to ICU admission, with 176 patients not admitted to the ICU and 34 patients admitted to the ICU.
2. Automated machine learning
In this study, we used the H2O-AutoML program to train the prediction model on the data and validated it through fivefold cross-validation, as shown in
Fig. 1. The performance is evaluated with 4/5ths of the training set and 1/5th of the validation data, and the average of the five performance results is called the final performance of the training model.
H2O-AutoML uses supervised learning, which consists of classification and regression and involves learning using a dataset that already knows the result
6. Classification refers to the process of identifying the categorical relationship between existing data and determining the category of newly observed data by itself, and there are algorithms such as decision tree and random forest. Regression is the process of averaging a dependent variable over an independent variable.
3. Statistical analysis
The area under the receiver operating characteristic (AUROC) curve was used to compare the ML models. The model with the highest AUROC in this study was the generalized linear model (GLM), which extends the distribution of the response variable from a traditional linear regression model to multiple distributions, including a normal distribution. This model generalizes linear regression by relating it to the response variable through a link function.
Statistical analyses were performed using IBM SPSS Statistics (ver. 29.0.0.0; IBM).
III. Results
GLM had the highest AUROC values in models 1 and 2, exhibiting very good diagnostic accuracy with an area under the curve ≥0.8. In model 1, the training set had an AUROC of 0.9191.(
Fig. 2. A) In the cross-validation set, the AUROC was 0.8289.(
Fig. 2. B) Variable importance analysis, shown in
Fig. 3, revealed that the CRP level was the most important variable, followed by DM, odynophagia, dysphagia, neutrophil count, and mandibular posterior infection.
In model 2, the training set had an AUROC of 0.9263.(
Fig. 4. A) In the cross-validation set, the AUROC was 0.8415.(
Fig. 4. B) The variable importance analysis, shown in
Fig. 5, revealed that the CRP level was the most important variable, followed by DM, neutrophil count, submental space, and submandibular space.
IV. Discussion
Although model 2 was expected to be superior to model 1, we wanted to independently determine this conclusion. This study demonstrated that models can predict the early ICU admission of patients with oral infections using ML algorithms. The suggested model was formulated using the initial assessment and examination results upon admission, offering the benefit of promptly predicting ICU admission shortly after patient admission.
Model 2 demonstrated a higher AUROC value compared to model 1. However, both were good models with AUROC values >0.8. Therefore, we expect that model 1, which does not consider enhanced MDCT images, can predict the need for early ICU admission.
The comparison of variable importance revealed that CRP was the most important parameter in both models 1 and 2. CRP originates in the liver and is released into the bloodstream, and elevated CRP levels swiftly indicate the presence of inflammation or infection, allowing the CRP blood test to assess the extent of inflammation. Therefore, CRP level can be an important clue for determining ICU admission.
The next most important variable was DM. Some studies determined that individuals with DM have a twofold increase in the likelihood of hospitalization upon emergency room admission for an infection. Additionally, infections account for up to 12% of hospital admissions in patients with DM
7. A research study revealed that individuals with DM have a considerably higher risk of being admitted to the ICU and experiencing mortality when hospitalized for infections affecting the skin or soft tissue, central nervous system, or bone and joints
8. In individuals with DM, head and neck infections are prevalent and tend to be more severe. A study conducted at the National Taiwan University Hospital focusing on deep neck infections revealed that individuals with DM exhibited a notably higher incidence of abscess formation compared to those without DM (89.3% vs 71.3%). Moreover, surgical drainage was determined to be a more frequent necessity in patients with DM (86% vs 65.2%, respectively)
9. Research has shown that DM is not only linked to an increased risk of infection, but is also associated with increased rates of hospitalization, ICU stay, and mortality resulting from these infections
10. It is important to highlight that many individuals with DM often have additional health conditions, and fully accounting for these factors may pose challenges in epidemiological studies, potentially leading to higher estimates of the infection risk associated with DM.
The next most important parameters were odynophagia, dysphagia, and neutrophils in model 1 and neutrophils, submental space, and submandibular space in model 2. However, considering that submental and submandibular space abscesses cause odynophagia and dysphagia, the parameters that were important in both models corresponded to each other.
This study has several limitations. First, we collected data from a single center. Therefore, the sample size was not sufficiently large. Only 210 patients met the inclusion criteria over the 10-year period. In addition, because the ML algorithm deduces outcomes by simultaneously considering multiple factors, generating a precise singular value for each variable is unattainable.
This study demonstrates the efficiency of ML models in the early prediction of ICU admission by exclusively utilizing initial examination findings. To ensure uniform and objective data collection, we systematically and impartially gathered data from patients with oral infections who sought treatment at our hospital from March 2013 to August 2023. A notable strength of our models is their sole reliance on clinical examinations and blood tests during the initial visit, eliminating the need for imaging. These attributes enable surgeons to make appropriate decisions.
Predictive models can help clinicians make early decisions regarding ICU admission based solely on blood test results or clinical examinations of patients with oral infections, even when enhanced MDCT imaging is not available. In the coming years, artificial intelligence-driven applications have the potential to enhance the quality of patient care and streamline the development of advanced decision-making tools. Subsequent research should aim to create a computerized system capable of gathering, digitizing, and sharing extensive data, such as blood test results and clinical examinations, from various institutions.
V. Conclusion
We demonstrate that ML has the potential to predict the likelihood of ICU admission in individuals with oral infections. Model 2 is better than model 1, but considering its ease of use, model 1 seems to be useful in clinical practice.