A multicenter-based, retrospective, consecutive cohort study from June 2019 to December 2020 was conducted at five hospitals in Hallym Medical Center. Patients were classified as high risk with the National early warning score (NEWS) over seven regardless of the single item score. After RRS activation, rapid response team members were immediately summoned to assess the patient's condition. Also, they prescribed appropriate medications or escalated the level of care based on the clinical assessment. When the RRS was activated at different times in the same patient, we considered those activations as separate cases.

Of the 18,131 patients identified by RRS activations, we excluded 480 identified after the ICU admission. We further excluded 6,154 with missing data and 38 with incorrectly recorded NEWS scores. Finally, 11,459 patients with RRS activations were included in the analysis. Comorbidities were defined using the 10^th International Classification of Disease codes and consisted of cardiovascular disease, pulmonary disease, gastrointestinal disease, cerebrovascular disease, genitourinary disease, and cancer.9 The outcome of this study was unexpected ICU admission within 24 hours after the RRS activation. Data were described as medians (interquartile ranges) for continuous variables and numbers (percentages) for categorical variables. In high-risk patients, factors relating to unexpected ICU admission were further evaluated using the multivariable logistic regression analysis. All factors were included in the analysis. Statistical analyses were performed using R 4.0.3 (R Foundation for Statistical Computing, Vienna, Austria).

Clinical characteristics of the RRS activation events were described in Table 1. Of 11,459 patients, the incidence of unexpected ICU admission was 7.4%, and the mean RRS activation was 3.1 times per patient. Patients who underwent unexpected ICU admission were heavier (22.2 [19.5–25.1] vs. 21.4 [18.5–24.2] kg/m², P < 0.001) and had higher total NEWS score (9 [8–10] vs. 8 [7–9] kg/m², P < 0.001) than patients who did not undergo unexpected ICU admission. Significantly different NEWS parameters were observed between two groups, except systolic blood pressure (P = 0.051); heart rate (103 [90–130] vs. 100 [100–110] rates/min), respiratory rate (25 [22–30] vs. 22 [21–26]), body temperature (37.1 [36.6–37.8] vs. 36.9 [36.6–37.5]), oxygen saturation (91 [87–950] vs. 95 [91–96]), need for supplemental oxygen (79.1% vs. 89.2%), and altered mental status (0.8% vs. 0.2%). Among etiologies, there was a low proportion of pulmonary disease (36.2% vs. 48.9%), genitourinary disease (6.6% vs. 4.0%), cerebrovascular disease (4.0% vs. 2.1%), and cancer (19.8% vs. 39.2%) in patients who underwent unexpected ICU admission. This difference was also confirmed in the NEWS score.

In the analysis for exploring factors relating to unexpected ICU admission, body mass index, pulmonary disease, and cancer were associated with the decreased risk of unexpected ICU admission (Fig. 1). Of NEWS parameter scores, high scores of systolic blood pressure, heart rate, respiratory rate, oxygen saturation, and level of consciousness were related to the increased risk of unexpected ICU admission. However, a high oxygen supplement score was associated with the decreased risk of unexpected ICU admission.

We found distinct clinical characteristics in patients who underwent unexpected ICU admission among high-risk patients identified by RRS. All NEWS scores were associated with the unexpected ICU admission, except body temperature. Low body mass index, pulmonary disease, and cancer were associated with the decreased risk of unexpected ICU admission.

Among high-risk patients identified by the RRS, there were differences in clinical characteristics according to the experience of unexpected ICU admission. In particular, almost all NEWS parameters were worse in patients who underwent unexpected ICU admission. It means that even among detected high risk patients, there are more severe patients who required immediate management.10 Our finding suggested that it will be possible to prioritize patient management in high risk patients identified by the RRS. Interestingly, patients who underwent unexpected ICU admission were heavier and had a low proportion of pulmonary disease and cancer. It may be because a large number of patients with pulmonary disease and cancer were chronically ill. In these patients, the heart rate or respiratory rate was high because of their underlying conditions.1112 Also, they are malnourished and often require long-term oxygen therapy.1314 The discrimination ability of RRS for identifying high risk patients may be insufficient in chronic patients who had a high baseline early warning score. Therefore, the etiology of the patient must be considered in the RRS implementation.15

Most of the high NEWS scores were associated with the increased risk of unexpected ICU admission. Notably, systolic blood pressure showed no difference between those who underwent unexpected ICU admission and those who did not. However, systolic blood pressure was related to the increased risk of unexpected ICU admission. On the other hand, an inversed relationship was observed on the body temperature. The body temperature, which was higher in those who underwent unexpected ICU admission, did not show significance in exploring risk factors. These results indicate that systolic blood pressure is a more important parameter for predicting acute deterioration than body temperature in managing high-risk patients. A previous study has reported that adverse events can be accurately predicted using systolic blood pressure added to other indicators without body temperature.1617 Therefore, clinicians should cautiously interpret the details of NEWS parameters.

There are some limitations to this study. First, a causal relationship could not be confirmed because this is a retrospective study. Also, this study was conducted based on the retrospective review of the patient's records. Thus, some parameters, such as level of consciousness, might be inaccurate. Despite these limitations, we should address the strength of this study. First, to the best of our knowledge, this is the first study that reported characteristics and prognosis of the high-risk patients based on the results of the RRS pilot program in Korea. Also, our study can be used for prioritizing management among a large number of high-risk patients identified by RRS. In a resource-limited environment such as Korea, prioritizing patients who need immediate management will be of great help in the operation of RRS.18

In conclusion, there were differences in clinical characteristics among high risk patients, and those differences were associated with unexpected ICU admissions. Clinicians should consider factors relating to unexpected ICU admission in the management of patients identified by the RRS.