PDF
Citation
Print
INTRODUCTION
Predicting mortality is very important in the process of caring for critically ill patients. Depending on the likelihood of mortality, the urgency of the use of medical resources can be assessed, and the medical condition can be detected and treated early before progression [1-3]. There are various tools for predicting mortality in critically ill pediatric patients, among which Pediatric Index of Mortality 3 (PIM 3) and Pediatric Logistic Organ Dysfunction-2 (PELOD-2) are widely used worldwide [4,5]. The predictive performance and effectiveness of both prediction tools have been proven through several validation studies [6-9].
However, in a retrospective study of children who received bone marrow transplantation, it was reported that there was no significant difference in mortality predicted by PIM (previous version of PIM 3) between survivors and non-survivors [10]. Another study of patients undergoing surgery for congenital heart disease reported a weak relationship between the severity of the patient’s condition and the PELOD (previous version of PELOD-2) score [11]. Since it is important to be aware that the performance of a mortality prediction system may vary according to a specific disease or patient group, we attempted to find studies on PIM 3 and/or PELOD-2, which are the upgraded versions of PIM and PELOD, respectively. However, to the best of our knowledge, none of the available studies exactly fit this purpose. Therefore, we designed this study with the aim of determining whether there are patient group characteristics that influence the mortality predictive performance of PIM 3 and PELOD-2, and if any, we aimed to determine the specific factors that cause the difference in performance between these tools.
MATERIALS AND METHODS
Study Setting
This retrospective observational study was conducted at a 24-bed medical and surgical pediatric intensive care unit (PICU) of a tertiary hospital. Patients under the age of 18 years who were admitted to the PICU from July 2017 to May 2019 were included, and patients with vital signs that were considered non-physiologic were excluded from the analysis. The non-physiologic vital signs were defined as: heart rate (HR) above 300 beats/min or below 30 beats/min, respiratory rate (RR) above 120 breaths/min or below 5 breaths/min, body temperature above 42°C or below 30°C, and oxygen saturation below 30%.
Data Collection and Pre-processing
The following data were collected from the hospital’s electronic health records: age; sex; physical findings such as blood pressure (BP), HR, and RR; clinical findings such as vasoactive-inotropic scores and the use of mechanical ventilation; and laboratory findings such as blood gas analysis results and electrolyte levels. Among these variables, BP, HR, and RR, whose normal ranges change with age [12], were not used in order to avoid age-related bias, but the z-score for each variable was calculated and used for analyses. In the process of calculating the z-score, the “generalized additive models for location scale and shape” and “sitar” package of R software (R Foundation for Statistical Computing, Vienna, Austria) were used [13,14]. The PIM 3 and PELOD-2 scores were calculated using formulas presented in the development studies [4,5]. When calculating the scores of the above tools, the worst one was used within 1 hour of entering the PICU, and the results of the examination within 1 hour of entering the PICU were used based on the examination execution time, not the examination result report time. The process of recording data through the hospital information system was performed by one researcher, and the PIM 3 and PELOD-2 scores were calculated through R coding.
Outcome Measures
The primary outcome in this study was an analysis of factors affecting the mortality prediction performance of PIM 3 and PELOD-2. For this, the value obtained by subtracting the predicted mortality rate of PELOD-2 from the predicted mortality rate of PIM 3 was defined as “difference in mortality prediction”, and related factors were analyzed using multivariable linear regression. The secondary outcome was whether there was a difference from observed mortality in each subgroup; this was obtained by performing subgroup analysis on categorical variables among factors that were significantly related to “difference in mortality prediction” in the multivariable analysis.
Statistical Analysis
To analyze the relationship between the mortality predicted by PIM 3 or PELOD-2 and the observed mortality, the area under the receiver operating characteristic curve (AUROC) and Hosmer-Lemeshow goodness of fit test were used, and the grade was set to 10 steps. Linear regression analyses were used to analyze factors related to the “difference in mortality prediction,” and factors that showed significant results in the univariable analyses were used to create a multivariable linear regression model. The final model was derived using the backward selection method. All statistical analyses were performed using R version 4.0.3 (R Foundation for Statistical Computing, Vienna, Austria), and P<0.05 were considered statistically significant.
RESULTS
Baseline Characteristics
During the study period, a total of 1,073 patients were screened, and 945 patients were finally analyzed after applying the inclusion and exclusion criteria (Figure 1). The median (interquartile range) age was 3.0 years (0.0–8.0 years), and 44.7% of the patients were girls. Table 1 provides detailed information regarding the baseline characteristics. The Hosmer-Lemeshow goodness of fit test was conducted to confirm whether the mortality predicted by the severity scoring tools differed from the observed mortality. The results of PIM 3 (AUROC=0.889, χ2=10.187, P=0.313) showed no statistically significant difference from the observed mortality. The results of PELOD-2 were also AUROC=0.731, χ2=6.220, P=0.183, showing that there was no difference from the observed mortality. Both PIM 3 and PELOD-2 showed fair to good predictive performance in predicting the observed mortality (Figure 2).
Main Outcomes
Based on the multivariable analysis, oxygen saturation (β=–0.065, standard error [SE]=0.012, P<0.001), base excess (β=–0.124; SE=0.024, P<0.001), a diagnosis of seizures (β=–3.598, SE=0.723, P<0.001), and cardiac bypass surgery (β=–2.083, SE=0.264, P<0.001) were associated with a decrease in the “difference in mortality prediction” between the tools. That is, the mortality rate predicted by PELOD-2 tends to increase to a greater extent than the mortality rate predicted by PIM 3 as the factors correspond to the above variables. In contrast, partial pressure of CO2 (β=0.041, SE=0.010, P<0.001); platelet counts (β=0.004, SE=0.001, P<0.001); blood urea nitrogen levels (β=0.045, SE=0.017, P=0.008); diagnoses of cardiomyopathy or myocarditis (β=3.810, SE=0.948, P<0.001), necrotizing enterocolitis (β=4.356, SE=1.356, P<0.001), cardiac arrest (β=20.691, SE=0.813, P<0.001), and leukemia or lymphoma after the first induction (β=9.066, SE=2.163, P<0.001); bone marrow transplantation (β=6.255, SE=1.542, P<0.001); and liver failure (β=5.907, SE=1.257, P<0.001) were associated with an increase in the “difference in mortality prediction” (the more the above factors were met, the higher the predicted mortality rate of PIM 3 was that of PELOD-2) (Table 2).
Table 3 shows the results of the subgroup analyses based on categorical variables among the factors confirmed to affect “difference in mortality prediction” in the multivariable analysis. Similar to the results of the multivariable analysis, PELOD-2 had a higher predictive mortality rate than PIM 3 in cases of bypass cardiac surgery or seizures, and PIM 3 had a higher predictive mortality rate in other cases. In the Hosmer-Lemeshow goodness of fit test result, only the predicted mortality of PIM 3 could be analyzed in both ‘cardiomyopathy or myocarditis’ and ‘cardiac arrest’ cases, and none of them showed a statistically significant difference; thus, PIM 3 predicted the actual observed mortality well in the subgroup (Table 3).
DISCUSSION
We conducted this study to determine whether there may be a difference in the predictive performance between PIM 3 and PELOD-2. Further, we investigated the specific factors that cause the difference. We found that both PIM 3 and PELOD-2 showed good performance in predicting the observed mortality; however, both showed slightly different results in predicting mortality according to the clinical characteristics of the patients.
Previous studies reported that the AUROC range of PIM 3 was 0.75–0.88 [8,15-19]. The AUROC of PELOD-2 was reported to be in the range of 0.75 to 0.94 [7,17-20]. In our results, the AUROC values of PIM 3 and PELOD-2 were 0.889 and 0.731, respectively, which were not significantly different from those in previous studies.
In this study, the “difference in mortality prediction” was affected by several factors, suggesting that the predicted mortality rates of PIM 3 and PELOD-2 may be affected by the characteristics of the patient group. As mentioned earlier, there is no existing study comparing PIM 3 and PELOD-2 according to patient group characteristics; thus, it was impossible to directly compare the results of our study with those in the existing literature. However, we were able to find one published paper suggesting that PELOD-2 scores may be lower in certain patient groups [20]. That study is a prospective observational study of critically ill children who needed plasma transfusion admitted to 101 PICUs in 21 countries. It was reported that the mortality prediction of PELOD-2 showed a fair performance, i.e., an AUROC of 0.76, but a relatively low predictive power compared to previous results, i.e., an AUROC of 0.934. In addition, the study concluded that the predictive power of PELOD-2 may be different in specific subpopulations [21]. Although plasma transfusion itself was not analyzed as a relevant factor in our study, PELOD-2 showed a lower mortality rate than PIM 3 in patients with leukemia, bone marrow transplantation, and liver failure who were expected to require large amounts of plasma transfusion. Of course, these results cannot be directly applied; however, we believe it might be a worthwhile point considering the relevance to existing studies conducted in patients who received plasma transfusions.
In the subgroup analysis results, most observed mortality was closer to the predicted mortality of PIM 3 than the predicted mortality of PELOD-2. However, we did not think that this was a result that meant that PIM 3 was superior to PELOD-2. This is because, while targeting a specific subpopulation, the sample size corresponding to each subgroup was very small (e.g., only three patients in the case of leukemia or lymphoma after first induction). Additionally, since this subgroup analysis is binary based on whether the listed factors are or not, there may be other potential confounding factors in each subgroup. Further, as a result of the Hosmer-Lemeshow test, most of the items were not applicable. The test was designed to compare the predicted mortality rate range (from 0% to 100%) with mortality by dividing it into 10 segments with 10% intervals. Therefore, it was difficult to derive results from the small sample population by dividing it into several subgroups.
This study has several limitations. First, this was a single center study. Thus, there may be differences in the results when the tools are applied in other institutions. However, we attempted to include a sufficiently large number of participants in our study. Second, the timing at which each tool was applied may have been different for individual patients. This is because the calculation definitions of PIM 3 and PELOD-2 are different, and our study was performed within a defined calculation time range. Third, the sample size was relatively insufficient to perform subgroup analysis. Finally, it was thought that clinical management before ICU admission might have been different for each patient, but it is not considered to be the focus of the study itself.
Both PIM 3 and PELOD-2 showed good results in predicting mortality but showed different predictive results depending on the specific clinical characteristics of the patient. Since the study was conducted at a single center and contained a relatively insufficient sample size, it may be difficult to directly apply the results of this study to other institutions. Therefore, it is necessary to supplement these results with multicenter studies including sufficient sample sizes in the future. Moreover, when applying and interpreting the above tools in clinical practice based on these results, it is necessary to consider the characteristics of each individual patient.
XML Download