PDF
Citation
Print
I. Introduction
Recently, the medical field has begun developing machine learning (ML) models that can aid doctors in their decision-making regarding diagnosis, prognosis, and management of multifactorial diseases1-3. The dentistry field is using ML to improve patient care1,3-5. For example, a deep-learning-based convolutional neural network algorithm was used to diagnose dental caries in periapical radiographs4, and neural network ML considering several cephalometric variables was used to diagnose the need for extractions in the field of orthodontics5. Additionally, various studies using artificial intelligence (AI) have been published in the fields of oncology and periodontics3.
As a subtype of AI, ML extrapolates results based on data and algorithms. The algorithms are trained using data with previously determined input-output relationships to infer output data with high predictability for new input data6,7. As the demand for ML increases, automated ML programs that can be used effectively by non-experts are available for free or a nominal fee (e.g., DataRobot, H2O-AutoML, H2O-Driverless AI product, Darwin, Google Cloud AutoML, TPOT)8.
Medication-related osteonecrosis of the jaw (MRONJ) is a multifactorial disease. Although medication is one of the main risk factors, local, demographic, and systemic factors also affect its occurrence9. Medications that can influence MRONJ occurrence include antiresorptive therapies such as bisphosphonates (BPs), denosumab (DMB), and romosozumab, which are administered orally or parenterally to manage osteoporosis, cancer-related bone metastases, and metabolic bone diseases. MRONJ risk in patients with osteoporosis ranges from 0.02% to 0.3%, which is not higher than that in patients taking drugs for other diseases9. However, MRONJ can reduce quality of life, and it is even life-threatening in some cases10; hence, early diagnosis and treatment are critical. It is important to not only accurately diagnose MRONJ in a timely manner, but also to predict potential occurrence and provide appropriate follow-up during the perioperative surgical period.
The aim of this study was to develop and validate ML models that can predict MRONJ occurrence with information obtained from a questionnaire. We used an automated ML program, H2O-AutoML, to develop models for patients with osteoporosis who had undergone tooth extraction or implantation surgery.
II. Patients and Methods
1. Patients
This study included female patients aged 55 years or older who visited the Department of Oral and Maxillofacial Surgery at Dankook University Dental Hospital between January 2019 and June 2022. Medical and dental history surveys were conducted using a questionnaire presented in Table 1, with a focus on the duration and administration of antiresorptive medications for patients with osteoporosis. The inclusion criteria were as follows: female, age ≥55 years, osteoporosis currently or previously treated with antiresorptive therapy (e.g., BPs or DMB), and recent dental extraction or implantation (e.g., patients who were referred after surgery at the local center and patients who had undergone surgery at this hospital). The exclusion criteria were radiotherapy history for head and neck cancer and incomplete medical and dental history records. A total of 340 patients met the inclusion criteria and was included in the study. We considered medication, demographic, systemic, and dental factors. Medication factors included duration and administration of antiresorptive medications (oral [PO], intravenous [IV], or both). Demographic and systemic factors included age and history of hypertension, hyperlipidemia, diabetes mellitus, heart disease, thyroid disease, kidney disease, liver disease, rheumatoid arthritis, cancer, or dementia. Additionally, any history of anticoagulant medication was considered. We also included local factors, such as the method of operation (extraction or implantation), number of operated teeth, and operation area (maxilla, mandible, or both).
MRONJ occurrence was evaluated during post-surgical follow-up after at least six months and was diagnosed according to criteria defined in the position paper of the American Association of Oral and Maxillofacial Surgeons9:
(1) Current or previous treatment with antiresorptive therapy alone or in combination with immune modulators or antiangiogenic medications.
(2) Exposed bone or bone that can be probed through an intraoral or extraoral fistula(e) in the maxillofacial region that has persisted for more than eight weeks.
(3) No history of radiation therapy to the jaws or metastatic disease to the jaws.
Because this study aimed to predict only MRONJ occurrence, the staging system (stages 0-3), which is helpful for doctors in determining a patient’s prognosis and outcome, was not considered. Instead, the output value was dichotomized as presence or absence of MRONJ.
Participants were divided into a control group (n=210) and MRONJ group (n=130). A comparison of the parameters between the control and MRONJ groups is presented in Table 2.
2. Automated ML
The H2O-AutoML program was used to perform hyperparameter optimization and train the prediction models2. The data were randomly divided into a training set (71.5%, n=243) and a testing set (28.5%, n=97). The training set was used to generate the prediction model, which was estimated through fivefold cross-validation. The testing set was used to estimate the accuracy of the models. The flow diagram of the proposed work is shown in Fig. 1.
ML can be classified into two categories: supervised and non-supervised. Supervised learning uses known labeled datasets to train an algorithm. Non-supervised learning uses unlabeled datasets11. Among these, supervised learning, particularly classification, is suitable for disease prediction because it involves discrete output variables. Thus, many studies have used this method1-3,11,12. We used the six algorithms of the generalized linear model (GLM), distributed random forest (DRF), gradient boosting machine (GBM), stacked ensemble, extreme gradient boosting (XGBoost), and deep learning to generate our prediction model.
The GLM is a regression model that estimates outcomes following exponential distributions. Although this model is easy to use and interpret, it cannot predict complex problems13. In contrast to using a single model like GLM or decision tree (DT) algorithms, the ensemble method uses numerous models to increase forecast accuracy12. The three combination strategies of the ensemble method are bagging, boosting, and stacking.
Bagging creates multiple classifications, trains them with different training data (bootstrapping), and then votes on different predictions for the same test data (aggregating)6,14. In this study, we used the DRF method to bag the DT algorithm.
While bagging learns several classifiers based on different algorithms in parallel, boosting trains classifiers of the same algorithm and learns from the errors of previous iterations by increasing the importance of incorrectly predicted training instances in future iterations6. GBM and XGBoost are examples of such boosting ensembles.
Stacking involves two phases. The first phase trains the dataset base using different algorithms, and the output of this phase is used as a new dataset for the second phase, called meta-learning, which provides the final output6,14.
Deep learning is a subtype of ML that utilizes a representation-learning method to enable a machine to find the necessary representations required for detection or classification from raw, abstract data such as pictures. This process is not designed by human engineers but learns itself from data using a common process15.
3. Statistical analyses
Statistical analyses were conducted on continuous and categorical variables between the control and MRONJ groups using Student’s t-test and the chi-square test, respectively, as appropriate, following the Kolmogorov–Smirnov test. P-values <0.05 were considered statistically significant. The area under the receiver operating characteristic (ROC) curves (AUC) and F1 score were used for comparison between ML models. Statistical analyses were performed using IBM SPSS Statistics (ver. 29.0.0.0; IBM).
III. Results
Of the 29 models developed for predicting MRONJ, 20, including 14 GBM, 4 stacked ensemble models, 1 DRF model, and 1 XGBoost model, showed very good diagnostic accuracy with an AUC ≥0.8. The GBM model had the highest AUC value. The training and test dataset metrics for the model are presented in Tables 3 and 4, respectively. This model comprised 42 trees with a maximum depth of 6 and showed an AUC of 0.8283 on the training dataset.(Fig. 2. A) At the maximum F1 threshold, the sensitivity and specificity were 83.7% and 71.3%, respectively. For the validation dataset, the model showed a stable AUC of 0.7526.(Fig. 2. B) At the maximum F1 threshold, the sensitivity and specificity were 88.6% and 52.8%, respectively.
Variable importance analysis, shown in Fig. 3, revealed that the duration of medication was the most important variable, followed by age, number of operated teeth, operation area, hypertension, and administration of medication.
IV. Discussion
Korea has a rapidly aging population; thus, a large proportion of the population has potential for osteoporosis16. BPs are frequently prescribed for female patients diagnosed with osteoporosis after menopause because they are non-invasive and easy to take orally. However, with age, the need for tooth extraction and implantation increases. Unfortunately, life-threatening MRONJ could occur after a seemingly simple dental treatment, and it cannot be easily predicted due to limited available information. Although biomarkers such as the C-terminal telopeptide of type-I collagen obtained through blood tests have been used in MRONJ clinical decision-making, their use remains controversial. While a drug holiday can be considered to reduce MRONJ risk in patients undergoing tooth extraction or implant surgery, it is not always feasible due to potential side effects such as skeletal-related events, fragility fractures, increased rebound bone resorption, or in emergency surgery situations. Recommendations for the duration of a drug holiday also remain controversial9. Recently, parathyroid hormone therapy, which stimulates new bone formation by osteoblasts, has been highlighted as an alternative drug. However, its use is limited to 24-month regimens due to potential side effects, and the subcutaneous daily injections required could be challenging for elderly patients17.
This study demonstrates the effectiveness of ML models for predicting MRONJ occurrence using only questionnaire-based information in patients with osteoporosis who have undergone tooth extractions or implant surgery. We objectively and consistently collected information from patients using BPs who visited our hospital in 2017 using the same questionnaire, which allowed for consistent and objective data collection. One advantage of our models is that they were developed based only on the questionnaire at the first visit, without the need for imaging or laboratory tests. These features enable surgeons to make clinical decisions and educate patients with more accurate information.
Most models in this study showed good diagnostic accuracy, with an AUC ≥0.8. For the validation dataset, the model showed an AUC of 0.7526, which is slightly lower but still a good predictive value. Additionally, sensitivity increased from 83.7% to 88.6%, and specificity decreased from 71.3% to 52.8%. With reference to our model, sensitivity can be seen as a more important indicator than specificity in that our model is intended to predict the possibility of MRONJ occurrence for educating patients and for follow-up.
We also compared variable importance.(Fig. 3) The most important variable was duration of medication use, followed by age. In this study, the mean ages of the control and MRONJ groups were 73.1 and 77.2 years, respectively, and the mean durations of medication use were 3.4 and 5.8 years. Both variables were significantly higher in the MRONJ group than in the control group. Many studies have also found significant increases in incidence rates, from 0.04% to 0.21%, in patients with osteoporosis who had been using BPs for more than four years9,18,19. As osteoporosis in women generally occurs after menopause, age and duration of medication are closely related.
The second most significant variable was the number of teeth that underwent operation. Some studies have found that the MRONJ incidence rate was significantly higher in cases of multiple-tooth extractions than in single-tooth extractions20. However, our analysis showed that the average number of teeth extracted in the MRONJ group was lower than that in the control group, which contradicts previously reported results. In response to these results, other studies have reported that tooth extraction does not act as a local factor alone, but it is associated with the presence or absence of inflammatory dental disease9,19,20. Therefore, it is necessary to also consider the presence or absence of periodontal disease.
Several studies have reported on the surgical area associated with MRONJ, showing that MRONJ prevalence is two to three times higher in the mandible than in the maxilla9,19,20. The results of our study are consistent with previous studies, in that we found 27% and 69% prevalence in the maxilla and mandible, respectively.
The most important variable in patient medical history was hypertension. Despite several studies showing that hypertension, diabetes, and anemia significantly enhance the risk of MRONJ incidence9,21,22, in this study, hypertension was the most significant variable. Although the risk of overfitting in this algorithm cannot be completely ruled out because the variable distribution is not uniform, the association between MRONJ and hypertension has not been well studied, and further research is needed.
This study has several limitations. First, we collected the data from a single institution, and to include a large number of MRONJ cases (n=130, 38.2%) compared with the low incidence, we excluded drug holiday or steroid therapy from variables, which are also correlated with increased MRONJ risk9. Nevertheless, none of the variables showed a uniform distribution between the MRONJ and control groups. This may have caused overfitting or underfitting of the ML models. Second, the model was designed using only medical records based on patient questionnaires. Nonetheless, it is meaningful in that it created a model with a high AUC value. However, the accuracy of the model can be improved by considering variables from other records, including objective laboratory, genetics, or imaging data3. Finally, because the ML algorithm infers results by considering numerous factors at once, it is impossible to produce a single exact value for each variable.
A predictive model may help clinicians explain the possibility of MRONJ in greater depth to patients and help them actively cope with future complications before tooth extraction or implantation. Moreover, in the future, AI-based programs may support high-quality patient care and facilitate advanced decision tools. In future research, we anticipate developing a computerized system that can collect, computerize, and share big data from multiple institutions, including radiograph and laboratory data, in addition to medical history data.
V. Conclusion
We demonstrated that ML models can be used to predict the likelihood of MRONJ occurrence in patients with osteoporosis who have undergone tooth extraction or implantation surgery using information gathered from a questionnaire at their initial visit. Our findings suggest that the duration of medication use is the most significant variable associated with MRONJ risk, followed by age, number of operated teeth, and operation area.
XML Download