From February 2020 for one year we carried out a study of ICU patients. The study was conducted by the Critical Care Quality Improvement Research Center, affiliated with Shahid Beheshti University of Medical Sciences. It was performed in accordance with the Declaration of Helsinki. Ethics approval was obtained from the Institutional Review Boards of AJA University of Medical Sciences (No. IR.AJAUMS.REC.1397.697696). All participants or their companions signed written consents for using patients’ data in the analyses. Patients were from two teaching hospitals affiliated with Universities of Medical Sciences. The hospitals are well-equipped settings with high patient turnovers and are large referral and subspecialty centers.

We included all adult patients who were admitted to the ICU. For patients with readmission within the study time period, only the first ICU admission was considered for data entry and analysis. A number of demographic, clinical, and laboratory features were recorded for each patient. Also, patients were monitored for undergoing any important procedure, and for mortality during the ICU stay. The GCS scores were determined by four study anesthesiologists and intensivists who were experts in the assessment of patients. In each work shift, a study nurse recorded the information for each patient. Data were entered into a paper form, and then, into the spreadsheet of Microsoft Office Excel software.

The outcome of the study was mortality during ICU stay. The predictors were primarily selected based on a consensus of the authors considering the limits of time, budget, and personnel [15,16]. We recorded patients’ sex and age, significant comorbidities, vital signs, information on nosocomial infections if any, the reason for admission, arterial blood gas, and a number of other laboratory test results, GCS on admission or immediately before EI (for patients undergoing EI), and significant procedures (including EI). All recorded variables reflected the status of patients on admission (e.g., comorbidities) or within their ICU stay (e.g., blood pressure). All patients arrive at the ICU without EI. The EI variable was recorded as a binary feature rather than numeric. Therefore, patients with re-intubation during their ICU stay were also labeled 1. Data were assessed for variables’ distribution, missing values, duplicate cases, and severe class imbalance (less than 1% frequency for any level).

We used the random forest to estimate variable importance. The random forest was used for feature selection because of its good performance, low overfitting, easy interpretability, and robustness to the presence of correlated features [17]. The data were then partitioned into development (training) and validation (test) datasets; 80, and 20%, respectively. Next, we included the selected variables into a binary logistic model and evaluated the performance of the regression. Three logistic models were developed: a base model, the second model incorporating the interaction term of EI×GCS, and the third with added EI×GCS×admission category. The strength of associations was reported using adjusted odds ratios (ORs) with 95% confidence intervals (CIs). The model’s fitness was assessed with chi-squared tests and the amount of variation in the dependent variables explained by the models was evaluated using Nagelkerke R². We investigated the performance of classifiers by calculating accuracy metrics: the percentage of the correctly classified records, sensitivity, specificity, positive predicted value, F1, balanced accuracy, and area under the receiver operator characteristic. For statistical analyses, point estimates, 95% CI, and p-values were calculated. P-values less than 0.05 were considered significant. Results are presented as mean and standard deviation (SD) for continuous variables, and as absolute numbers (%) for categorical data. The means of the continuous variables were compared using independent sample t-tests. The normality of the outcome variables was examined with the Shapiro-Wilk test and the homogeneity of variances was investigated with Levene’s test. Either a chi-square test or Fisher’s exact test was used for testing differences among the study groups for categorical variables.

We imported the data into R software version 4.0.2 (A language and environment for statistical computing (R Foundation for Statistical Computing; https://www.R-project.org/). R is a well-known free software environment for statistical and machine learning libraries and graphics. We used a variety of R packages for the analysis. All the packages were downloaded from the Comprehensive R Archive Network (https://cran.r-project.org/), the official R package repository, or the GitHub (https://github.com/) website.

Data from 2,055 patients were analyzed. There were no missing data or duplicate cases in our sample. Mean age (SD) was 55.0 years (15.6 years) and 983 patients (47.8%) were women. Overall, 865 patients (42.1%) died and 893 (43.5%) underwent EI. A GCS of 10 was used as an indication for EI with no exceptions, all patients with GCS ≤10 underwent EI, no patients with GCS >11 underwent EI (Figure 1). Table 1 shows the main characteristics of dead and alive groups of ICU patients. There were statistically significant differences between the two groups in reason for admission, nosocomial infection, surgery, diabetes, myocardial infarction, chronic obstructive pulmonary disease, immunosuppression, use of anesthetics/sedatives/hypnotics for reasons other than EI, age, systolic blood pressure, and fraction of inspired oxygen (FiO₂) were significantly different between the two groups. Figure 2 shows the results of the feature selection. The features were sorted according to their importance to the prediction. We included the selected predictors in logistic regression models.

We constructed three logistic models of mortality: model 1 included the selected predictors, model 2 incorporated the interaction term of EI×GCS into the model 1, and model 3 added EI×GCS×admission category to model 2. The model 1, was well-fitted to the data, χ²(19)=693.817, P<0.001, Nagelkerke R²=0.534, Hosmer and Lemeshow goodness of fit test χ²(59)=2.464, P≈1.000, Akaike's information criterion (AIC)=1,305.1. The model 2 was also well-fitted, χ²(20)=708.370, P<0.001, Nagelkerke R²=0.523, Hosmer and Lemeshow χ²(59)=13.833, p≈1.000, AIC=1,292.5. But the model 3 was the best, χ²(32)=802.848, P<0.001, Nagelkerke R²=0.575, Hosmer and Lemeshow χ²(59)=10.755, P≈1.000, AIC=1,222.1. Also, pairwise comparisons between the models showed that model 2 is not statistically different from model 1, χ²(1)=2.626, P=0.105, ∆AIC=12.6; however, model 3 is better than model 2, χ²(12)=94.479, P<0.001, ∆AIC=70.4.

Analysis showed a high performance for model 3; accuracy (95% CI): 82.6% (79.4%–85.5%), McNemar's test P<0.001, sensitivity=90.4%, specificity=71.8%, positive predicted value=81.6%, F1=85.8%, balanced Accuracy=81.1%, and area under the receiver operator characteristic=0.81 (Figure 3). Table 2 shows the association of predictors with mortality. Overall, EI decreased the risk for mortality in ICU patients when adjusted for other risk factors. Patients with respiratory problems, nosocomial infection, diabetes, history of myocardial infarction, chronic obstructive pulmonary disease, immunosuppression, use of anesthetics/sedatives/hypnotics during ICU stay for reasons other than EI showed a greater risk for mortality. Also, death risk increased with increasing age and systolic blood pressure. In addition, GCS had plausibly an inverse relation with the risk of mortality. Of the model’s interaction terms, two were significant and two were not. The significant terms showed that in the “trauma” and “other” surgeries undergoing EI, GCS had an inverse relation with mortality compared with the reference category ("other medical problems"). However, in the “respiratory” problems and “elective brain” surgeries, patients undergoing EI did not show a significant relation between their GCS and the risk for mortality.