Journal List > J Korean Med Sci > v.37(16) > 1162096

TOOLS
Ko, Kwon, Cho, Lee, Kwon, Park, Kim, Kim, Jo, Lee, and Jeon: Quick Sequential Organ Failure Assessment Score and the Modified Early Warning Score for Predicting Clinical Deterioration in General Ward Patients Regardless of Suspected Infection
Original Article
Emergency & Critical Care Medicine

Journal of Korean Medical Science 2022; 37(16): e122.

Published online: 12 April 2022

DOI: https://doi.org/10.3346/jkms.2022.37.e122

Quick Sequential Organ Failure Assessment Score and the Modified Early Warning Score for Predicting Clinical Deterioration in General Ward Patients Regardless of Suspected Infection

Ryoung-Eun Ko1, *, Oyeon Kwon2, *, Kyung-Jae Cho2, Yeon Joo Lee3, Joon-myoung Kwon4, †, Jinsik Park5, Jung Soo Kim6, Ah Jin Kim6, You Hwan Jo7, Yeha Lee2, Kyeongman Jeon8

1Department of Critical Care Medicine, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Korea.

2VUNO, Seoul, Korea.

3Division of Pulmonary and Critical Care Medicine, Seoul National University Bundang Hospital, Seongnam, Korea.

4Department of Critical Care and Emergency Medicine, Mediplex Sejong Hospital, Incheon, Korea.

5Division of Cardiology, Cardiovascular Center, Mediplex Sejong Hospital, Incheon, Korea.

6Division of Critical Care Medicine, Department of Hospital Medicine, Inha University Hospital, Inha University College of Medicine, Incheon, Korea.

7Department of Emergency Medicine, Seoul National University Bundang Hospital, Seongnam, Korea.

8Division of Pulmonary and Critical Care Medicine, Department of Medicine, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Korea.

Address for Correspondence: Kyeongman Jeon, MD, PhD. Division of Pulmonary and Critical Care Medicine, Department of Medicine, Samsung Medical Center, Sungkyunkwan University School of Medicine, 81 Irwon-ro, Gangnam-gu, Seoul 06351, Korea. kjeon@skku.edu

Equal contributor:

*Ryoung-Eun Ko and Oyeon Kwon contributed equally to this work.

Current address:

†Present address: Medical AI, Co. Seoul, Korea.

Received 4 February 2022       Accepted 24 March 2022

© 2022 The Korean Academy of Medical Sciences.

The Korean Academy of Medical Sciences

https://creativecommons.org/licenses/by-nc/4.0/
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (https://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background

The quick sequential organ failure assessment (qSOFA) score is suggested to use for screening patients with a high risk of clinical deterioration in the general wards, which could simply be regarded as a general early warning score. However, comparison of unselected admissions to highlight the benefits of introducing qSOFA in hospitals already using Modified Early Warning Score (MEWS) remains unclear. We sought to compare qSOFA with MEWS for predicting clinical deterioration in general ward patients regardless of suspected infection.

Methods

The predictive performance of qSOFA and MEWS for in-hospital cardiac arrest (IHCA) or unexpected intensive care unit (ICU) transfer was compared with the areas under the receiver operating characteristic curve (AUC) analysis using the databases of vital signs collected from consecutive hospitalized adult patients over 12 months in five participating hospitals in Korea.

Results

Of 173,057 hospitalized patients included for analysis, 668 (0.39%) experienced the composite outcome. The discrimination for the composite outcome for MEWS (AUC, 0.777; 95% confidence interval [CI], 0.770–0.781) was higher than that for qSOFA (AUC, 0.684; 95% CI, 0.676–0.686; P < 0.001). In addition, MEWS was better for prediction of IHCA (AUC, 0.792; 95% CI, 0.781–0.795 vs. AUC, 0.640; 95% CI, 0.625–0.645; P < 0.001) and unexpected ICU transfer (AUC, 0.767; 95% CI, 0.760–0.773 vs. AUC, 0.716; 95% CI, 0.707–0.718; P < 0.001) than qSOFA. Using the MEWS at a cutoff of ≥ 5 would correctly reclassify 3.7% of patients from qSOFA score ≥ 2. Most patients met MEWS ≥ 5 criteria 13 hours before the composite outcome compared with 11 hours for qSOFA score ≥ 2.

Conclusion

MEWS is more accurate that qSOFA score for predicting IHCA or unexpected ICU transfer in patients outside the ICU. Our study suggests that qSOFA should not replace MEWS for identifying patients in the general wards at risk of poor outcome.

Graphical Abstract

jkms-37-e122-abf001.jpg

Keywords: Early Warning Scores, Modified Early Warning Score, Quick Sequential Organ Failure Assessment, Rapid Response System

INTRODUCTION

Some hospitalized patients experience clinical deterioration associated with hospital mortality or unexpected intensive care unit (ICU) transfers, which can result in increased morbidity and mortality.12 Previous studies postulate that patients show signs of increased risk hours before clinical deterioration; consequently, these early signals can be captured by regularly measured vital signs.123 Therefore, the early warning score (EWS) based on the vital signs was developed and adopted for screening patients with a high risk of clinical deterioration.12345 The Modified Early Warning Score (MEWS) based on five physiological parameters is an evaluation tool for the early identification of medical patients who needed intensive care.6
Sepsis, an inflammatory response to infection, contributed to ≥ 50% hospital mortality.7 The Sepsis-3 task force recommends the quick sequential (sepsis-related) organ failure assessment (qSOFA) score to rapidly identify patients with suspected infection at high risk of poor outcomes.8910 The qSOFA score, which is similar to the EWSs, was used in the general wards for screening and alerting when a patient is at high risk of a serious adverse outcome, irrespective of the underlying diagnosis.61112 However, the recent Surviving Sepsis Campaign recently recommended that qSOFA not be used as a single screening tool for sepsis or septic shock compared to the Systemic Inflammatory Response Syndrome criteria or MEWS.13
Introducing qSOFA in hospitals already using MEWS might pose a challenge because of the similar physiological variables and different weighting thresholds. Consequently, using both screening tools can lead to unnecessary duplication of staff effort, needless protocol complexity, and increased educational requirements. Recent publications demonstrated that MEWS discriminates in-hospital mortality, ICU admission, and their combined outcomes was better than qSOFA in patients with suspected infection.1415 However, data on the comparison in unselected hospital admissions to highlight the benefit of introducing qSOFA in hospitals already using MEWS are limited. Therefore, the aim of this study was to compare the performance of qSOFA as an early warning score with MEWS in hospitalized non-ICU patients regardless of diagnosis of infection by using a large database of routinely collected vital signs from five different characteristic hospitals in Korea.

METHODS

Study design and population

Over a 12-month period, a retrospective cohort study was conducted on all consecutive adult patients admitted to the general ward of five hospitals—Mediplex Sejong Hospital (a 323-bed secondary cardiovascular-specific hospital in Bucheon, South Korea, from January 2017 to December 2017), Sejong General Hospital (a 301-bed secondary cardiovascular-specific hospital in Inchon, South Korea, from March 2018 to March 2019), Inha University Hospital (a 925-bed university-affiliated, tertiary referral hospital in Inchon, South Korea, from January 2018 to December 2018), Seoul National University Bundang Hospital (a 1324-bed university-affiliated, tertiary referral hospital in Seoul, South Korea, from January 2017 to December 2017), and Samsung Medical Center (a 1989-bed university-affiliated, tertiary referral hospital in Seoul, South Korea, from July 2017 to June 2018). Each admission was collected separately for analysis in patients with multiple hospital admissions during the study period. Patients who met the following criteria were excluded: 1) admission directly to the ICU at hospital admission, 2) no vital sign data recorded in the 24 hours before in-hospital cardiac arrest (IHCA) or unexpected ICU transfer, or 3) only vital sign data of < 30 minutes recorded.

Data collection

Databases of vital signs collected from consecutive hospitalized patients older than 18 years over 12 months were created from five participating hospitals. The data was extracted using an each institution’s clinical data warehouse, which automatically retrieved data from electronic medical records. All of time-stamped basic vital signs including systolic blood pressure (SBP), diastolic blood pressure (DBP), heart rate (HR), respiratory rate (RR), body temperature (BT), and level of consciousness were collected during the hospitalization of the patients. The exact time and location of event occurrences were also extracted from the electronic medical records. Levels of consciousness were collected using the AVPU scale (alert [A]; responds to voice [V]; responds to pain [P]; or unresponsive [U]).1617 In addition, we categorized patients scoring V, P, or U as having “altered mental status” when calculating qSOFA scores.1819 From the initially collected data, erroneous values which are extremely outside of the acceptable range of each vital sign (30 to 300 mm Hg of SBP, 40 to 120 mm Hg of DBP, 10 to 300 beats/min of HR, 3 to 60 breaths/min of RR, and 30 to 45°C of BT) or non-numeric values were excluded and assumed to be missing values. Missing values are processed by using the most recent values measured before the time the missing value existed. Subsequently, the MEWS and qSOFA score were calculated. The performance of MEWS and qSOFA were compared to predict the outcomes within 24 hours of vital sign observation.
The primary outcome of the study was clinical deterioration defined as the composite of IHCA and unexpected ICU transfer. IHCA was defined as whether cardiac pulmonary resuscitation was performed during general ward admission in the electronic medical record. Unexpected ICU transfer was defined as an event wherein patients were transferred to the medical ICU that did not originate from either the emergency department or an operating room.2021 The secondary outcome of the study was IHCA and unexpected ICU transfer, respectively.
The qSOFA criteria were defined as SBP ≤ 100 mmHg, RR ≥ 22 breaths per minute, and altered mental status (defined as AVPU scale other than “Alert”).8 The MEWS was calculated based on previously published table (Supplementary Table 1).6

Statistical analysis

Data were presented as either median with interquartile range (IQR) or mean with standard deviation for continuous variables and number (%) for categorical variables. The discriminatory power of each score was assessed by calculating the area under each receiver operating characteristic curve (AUC). Accuracy comparisons were performed using sensitivity, specificity, positive and negative predictive values. The Youden index, defined as (sensitivity + specificity) − 1, was calculated to verify the current thresholds of qSOFA and MEWS. To calculate the AUC, true negative and false positive were calculated using each patient’s serial vital signs during their entire stay outside the ICU stay in patients who were not deteriorating clinically. In addition, each patient’s highest scores during their hospitalization were used in clinically not deteriorated patients, and each patient’s highest scores from 24 hours to 30 minutes before IHCA or unexpected ICU transfer were used in patients who deteriorated clinically. False positive and true negative were calculated using each patient’s serial vital signs from 24 hours to 30 minutes before IHCA or unexpected ICU transfer in patients who deteriorated clinically. Subsequently, the AUC for primary and secondary outcomes was calculated on the receiver operating characteristic curve, and values for the estimated AUC were compared using the McNeil test.22 To report predictive performance of qSOFA and MEWS at the critical threshold, the cutoff values of qSOFA ≥ 2 and MEWS ≥ 5 were used.68 Reclassification was evaluated using the net reclassification index (NRI),23 which quantifies the relative improvement of clinical deterioration prediction between MEWS ≥ 5 and qSOFA ≥ 2. The NRI is defined as (proportion of all events reclassified to clinical deterioration - proportion of all events reclassified to without clinical deterioration) - (proportion of all non-events reclassified to clinical deterioration - proportion of all events reclassified to without clinical deterioration). All analyses were performed using R Statistical Software (Version 3.5.1; R Foundation for Statistical Computing, Vienna, Austria). All tests of significance were two-tailed, and P < 0.05 was considered statistically significant.

Ethics statement

The Institutional Review Boards of all participating hospitals approved this study (Samsung Medical Center, SMC-2019-09-129; Seoul National University Bundang Hospital, B-2003-598-004; Inha University Hospital, 2020-02-013-000; Mediplex Sejong Hospital, 2018-054; Sejong General Hospital, 2018-0689) and waived the requirement for informed consent because of the observational nature of the research. In addition, this study adhered to relevant ethical guideline and the statement for reports of diagnostic accuracy studies with Standards for Reporting of Diagnostic Accuracy Studies (STARD).

RESULTS

Patient characteristics

During the study period, 173,057 patients hospitalized in the general wards, with available medical records and documented vital signs, were included. The baseline characteristics of the patients are summarized in Table 1. The mean age of the patients was 57.4 (± 15.8) years, and 86,549 (50.0%) patients were men. The median length of hospital stay was 5.9 days. At admission, the initial mean SBP, DBP, HR, RR, and BT was 126.5 (± 19.8) mmHg, 74.5 (± 12.3) mmHg, 77.8 (± 14.4) beats per minute, 18.1 (± 2.0) rates per minute, and 36.5°C (± 0.5°C). Of all patients, 755,724 (91.1%) were alert at admission. The prevalence of IHCA and unexpected ICU transfer was 1.29 per 1,000 admissions and 2.57 per 1,000 admissions, respectively. The baseline characteristics according to each of five hospitals are presented in Supplementary Table 2.
Table 1

Characteristics and clinical outcomes of hospitalized non-ICU patients

jkms-37-e122-i001
Variables Total (N = 173,057)
Age, yr 57.4 ± 15.8
Sex, male 86,549 (50.0)
Length of hospital stay, day 5.9 (1.6–6.7)
Vital signs at admission
Systolic blood pressure, mmHg 126.5 ± 19.8
Diastolic blood pressure, mmHg 74.5 ± 12.3
Heart rate, beat per minute 77.8 ± 14.4
Respiratory rate 18.1 ± 2.0
Body temperature, °C 36.5 ± 0.5
Mental status
Alert 755,724 (91.1)
Reacting to voice 29,123 (3.5)
Reacting to pain 9,676 (1.1)
Unresponsive 5,204 (0.6)
Clinical outcomes
In-hospital cardiac arrest 224
per 1,000 admission 1.29
Unexpected ICU transfer 444
per 1,000 admission 2.57
ICU = intensive care unit.

Comparison of qSOFA with MEWS

Algorithm discrimination for the clinical deterioration for MEWS (AUC, 0.777; 95% confidence interval [CI], 0.770–0.781) was higher than that for qSOFA (AUC, 0.684; 95% CI, 0.676–0.686; P < 0.001) (Fig. 1). The receiver operating characteristic curves of outcomes at each participating hospital are presented in Supplementary Figures. Besides, MEWS was better in predicting IHCA (AUC, 0.792; 95% CI, 0.781–0.795 vs. AUC, 0.640; 95% CI, 0.625–0.645; P < 0.001) and unexpected ICU transfer (AUC, 0.767; 95% CI, 0.760–0.773 vs. AUC, 0.716; 95% CI, 0.707–0.718; P < 0.001) than qSOFA. Most patients met the MEWS ≥ 5 criteria 13 hours before clinical deterioration compared with 11 hours for qSOFA ≥ 2 criteria (Fig. 2).
Fig. 1

The receiver operating characteristic curves of outcomes. (A) In-hospital cardiac arrest, (B) unexpected intensive care unit transfer, (C) composite outcome.

MEWS = Modified Early Warning Score, qSOFA = quick sequential organ failure assessment, AUC = areas under the receiver operating characteristic curve.
jkms-37-e122-g001
Fig. 2

Cumulative percentage of patients meeting ≥ 2 qSOFA score or ≥ 5 MEWS in the 24 hours before the outcomes. (A) In-hospital cardiac arrest, (B)unexpected intensive care unit transfer, (C) composite outcome.

MEWS = Modified Early Warning Score, qSOFA = quick sequential organ failure assessment.
jkms-37-e122-g002
Using the patient’s highest scores, the sensitivity and specificity of qSOFA ≥ 2 were 13.9% (95% CI, 13.3–14.5) and 98.6% (95% CI, 98.6–98.7) compared with those of MEWS ≥ 5 at 17.0% (95% CI, 16.1–17.7) and 99.4% (95% CI, 99.4–99.4) for clinical deterioration, respectively. The sensitivity, specificity, and positive and negative predictive values for the clinical deterioration at different cutoff points for each scoring system are presented in Table 2. The MEWS ≥ 5 resulted in reclassification of 3.7% of the patients from qSOFA ≥ 2. The NRI for clinical deterioration was 0.0033 (95% CI, 0.0002–0.0062) and the NRI for without clinical deterioration was 0.0073 (95% CI, 0.0072–0.0074; P = 0.043). The Supplementary Tables 3 and 4 show the predictive accuracy for IHCA and unexpected ICU transfer separately.
Table 2

Prediction accuracy for primary outcome according to different score thresholds

jkms-37-e122-i002
Score/threshold Sensitivity (%) Specificity (%) PPV (%) NPV (%) Youden index
qSOFA
≥ 1 53.9 80.3 0.3 99.9 0.3429
≥ 2a 13.9 98.6 1.3 99.8 0.1259
≥ 3 1.2 99.9 2.5 99.8 0.0116
MEWS
≥ 0 100 0 0.1 - 0
≥ 1 68.9 77.2 0.4 99.9 0.4600
≥ 2 48.3 93.6 0.9 99.9 0.4200
≥ 3 48.3 93.6 0.9 99.9 0.4201
≥ 4 28.9 98.0 1.8 99.9 0.2704
≥ 5a 17.0 99.4 3.5 99.8 0.1641
≥ 6 8.7 99.8 5.5 99.8 0.0860
≥ 7 3.7 99.9 6.4 99.8 0.0363
≥ 8 1.0 99.9 4.9 99.8 0.0098
≥ 9 0.4 99.9 5.6 99.8 0.0041
≥ 10 0.2 100.0 5.0 99.8 0.0015
≥ 11 0.0 100.0 0 99.8 0
≥ 12 0.0 100.0 0 99.8 0
≥ 13 0.0 100.0 0 99.8 0
≥ 14 0.0 100.0 0 99.8 0
≥ 15 0.0 100.0 - 99.8 0
PPV = positive predictive value, NPV = negative predictive value, qSOFA = quick sequential organ failure assessment, MEWS = Modified Early Warning Score.
aCommonly used cutoff thresholds.

DISCUSSION

The present study investigated the performance of qSOFA compared with MEWS for clinical deterioration in hospitalized non-ICU patients regardless of diagnosis of infection by using a large database of routinely collected vital signs from five different characteristic hospitals in South Korea. The qSOFA showed worse discrimination for clinical deterioration, IHCA, and unexpected ICU transfer than MEWS. In addition, qSOFA ≥ 2 showed lower sensitivity and specificity compared with MEWS ≥ 5.
The EWSs have been developed based on the findings that abnormal changes in physiologic parameters, such as vital signs or mental status, often precede overt clinical deterioration by several hours.324 The MEWS, and its derivatives, such as the national early warning score (NEWS), which was endorsed by the Royal College of Physicians for standard use across the United Kingdom, have been already used in many hospitals for the rapid response system.2526 MEWS includes SBP, HR, RR, BT, and level of consciousness, and values of 0 to 3 are given to each parameter based on the degree of abnormality.6 Several studies reported that recording of all vital signs in MEWS might be complex, and these complexities may be associated with adverse events from an incomplete set of observations.2728 Furthermore, there is debate regarding the performance of EWSs. Shappell et al.29 reported that NEWS has poor discriminatory power (AUC, 0.66; 95% CI, 0.66–0.67) for 28-day mortality. In addition, Ahn et al. also reported poor discriminatory power of MEWS (AUC, 0.58; 95% CI, 0.56–0.59) and NEWS (AUC, 0.60; 95% CI, 0.59–0.62), respectively.30
The Sepsis-3 task force proposed the qSOFA (also known as quick SOFA), an empirically derived score using simple clinical criteria, that may identify patients with suspected infection who are at greater risk for a poor outcome outside the ICU.8 It uses three criteria, assigning one point for low blood pressure (SBP ≤ 100 mmHg), high RR (≥ 22 breaths per minutes), or altered mentation (Glasgow Coma Scale < 15),8 which are not specific variables for infection. Compared with MEWS, qSOFA has the advantage of simplicity because it includes only three binary elements and does not require a reference table or calculator. However, the qSOFA was identified as a predictor of poor outcome in patients with known or suspected infection.8 Although numerous studies have investigated the potential use of the qSOFA as a screening tool for sepsis, the results have been contradictory to its usefulness.31 In addition, a retrospective register-based validation study demonstrated that qSOFA performs similarly in patients not yet diagnosed or suspected with infection, implying that qSOFA is not infection specific and could simply be regarded as a general EWS.32 These results support recent Survival Sepsis Campaign, which recommends against using qSOFA compared to other EWSs as a single screening tool for sepsis or septic shock.13
However, previous studies have shown that general EWSs are more accurate than qSOFA when predicting adverse outcomes in the general wards (Table 3).14151833 Churpek et al.14 compared the performance of qSOFA with MEWS and NEWS for predicting death and ICU transfer in patients with suspected infection at the emergency room or general ward in a single center during 8 years. They revealed that MEWS (AUC, 0.73; 95% CI, 0.71–0.74) and NEWS (AUC, 0.77; 95% CI, 0.76–0.79) were more accurate than qSOFA (AUC, 0.69; 95% CI, 0.67–0.70). Furthermore, Redfern et al.18 compared the performance of qSOFA with NEWS for predicting in-hospital mortality and unexpected ICU transfer in general ward patients regardless of infection in a single center during 6 years. They showed that NEWS (AUC, 0.813, 95% CI, 0.810–0.817) were more accurate than qSOFA (AUC, 0.671; 95% CI, 0.666–0.675). In the present study, MEWS was more accurate than qSOFA for predicting clinical deterioration in general ward patients regardless of suspected infection in five centers during 12 months. Taken together, these studies provide evidence that hospitals already using EWSs should not change to or introduce qSOFA for screening patients at risk of a poor outcome outside the ICU.
Table 3

Characteristics of studies comparing qSOFA and MEWS

jkms-37-e122-i003
Study Participating center Setting Duration of data collection Number of patients Admission Suspected infection MEWS (95% CI) qSOFA (95% CI) Primary outcome
Churpek et al.14 Single Urban tertiary academic center 111 months 30,677 (12,154 in ward and 18,523 in ED) Ward or ED Yes 0.73 (0.71–0.74) 0.69 (0.67–0.70) In-hospital mortality
Churpek et al.15 Single Urban tertiary academic center 111 months 53,849 Ward or ED Yes 0.67 (0.66–0.68) 0.65 (0.62–0.66) In-hospital mortality
Redfern et al.18 Single Tertiary general hospital 61 months 241,996 Ward Mixed 0.82 (0.82–0.82)a 0.68 (0.67–0.68) In-hospital mortality
Liu et al.33 Multi (32 hospitals in the USA) Mixed 156 months 1,487,263 Ward Mixed 0.83 (0.83–0.84) in California and 0.84 (0.84–0.85) in Illinois 0.78 (0.78–0.79) in California and 0.78 (0.77–0.78) in Illinois In-hospital mortality
Present study Multi (5 hospitals in the Korea) Mixed 12 months 173,057 Ward Mixed 0.77 (0.77–0.78) 0.68 (0.67–0.68) Clinical deterioration (in-hospital cardiac arrest and unexpected ICU transfer)
qSOFA = quick sequential organ failure assessment, MEWS = Modified Early Warning Score, ED = emergency department, ICU = intensive care unit, CI = confidence interval.
aNational Early Warning Score (NEWS).
Although this study provided additional information on the performance of qSOFA and MEWS for predicting clinical deterioration in hospitalized, non-ICU patients regardless of suspected infection, there are several limitations that should be acknowledged. First, the study was conducted in a single country, which may lessen the generalizability of our findings to other countries. However, we included various featured hospitals. Therefore, our results represent the actual performance of MEWS in non-ICU patients. Nevertheless, further comparisons in well-designed prospective studies are needed. Second, the use of electronic medical records data may also be limited by missing data resulting from incomplete documentation of certain clinical observations. Third, because only the vital signs of non-ICU patients were collected, comorbidity or reason for admission, including whether it was related to infection, were not investigated. In addition, detailed clinical information of patients such as comorbidities were not collected. Fourth, the single-payer national health system, which provides mostly private provision of healthcare in South Korea, has resulted in different hospitalization or ICU admission standards compared with other countries. Finally, our outcome included IHCA and/or unexpected ICU transfers, which had the potential of including expected IHCA and unexpected ICU transfers that did not reflect actual clinical deterioration. Although this composite outcome might reduce the clinical utility of risk scores in practice, it would unlikely influence the overall ordering of model discrimination.34
In this multicenter study, MEWS was more accurate than the qSOFA for predicting IHCA and unexpected ICU transfer in patients outside the ICU, suggesting that the qSOFA should not replace MEWS for identifying patients at risk of poor outcome in hospitalized patients in the general wards.

Notes

Funding: This work was supported by a Samsung Medical Center grant (SMO1200901).

Disclosure: The authors have no potential conflicts of interest to disclose.

Author Contributions:

  • Conceptualization: Ko RE, Kwon O, Jeon K.

  • Data curation: Ko RE, Kwon O, Cho KJ, Lee YJ, Kwon Jm, Park J, Kim JS, Kim AJ, Jo YH, Lee Y.

  • Formal analysis: Ko RE, Kwon O, Lee YJ, Kwon Jm, Park J, Kim JS, Kim AJ, Jo YH, Lee Y, Jeon K.

  • Investigation: Ko RE, Kwon O, Cho KJ, Lee YJ, Kwon Jm, Park J, Kim JS, Kim AJ, Jo YH, Lee Y.

  • Methodology: Ko RE, Kwon O.

  • Resources: Ko RE, Kwon O, Cho KJ.

  • Supervision: Jeon K.

  • Writing - original draft: Ko RE, Kwon O, Jeon K.

  • Writing - review & editing: Cho KJ, Lee YJ, Kwon Jm, Park J, Kim JS, Kim AJ, Jo YH, Lee Y, Jeon K.

References

1. Churpek MM, Yuen TC, Winslow C, Robicsek AA, Meltzer DO, Gibbons RD, et al. Multicenter development and validation of a risk stratification tool for ward patients. Am J Respir Crit Care Med. 2014; 190(6):649–655. PMID: 25089847.
2. Escobar GJ, LaGuardia JC, Turk BJ, Ragins A, Kipnis P, Draper D. Early detection of impending physiologic deterioration among patients who are not in intensive care: development of predictive models using data from an automated electronic medical record. J Hosp Med. 2012; 7(5):388–395. PMID: 22447632.
3. Kause J, Smith G, Prytherch D, Parr M, Flabouris A, Hillman K, et al. A comparison of antecedents to cardiac arrests, deaths and emergency intensive care admissions in Australia and New Zealand, and the United Kingdom--the ACADEMIA study. Resuscitation. 2004; 62(3):275–282. PMID: 15325446.
4. Churpek MM, Yuen TC, Edelson DP. Risk stratification of hospitalized patients on the wards. Chest. 2013; 143(6):1758–1765. PMID: 23732586.
5. Umscheid CA, Betesh J, VanZandbergen C, Hanish A, Tait G, Mikkelsen ME, et al. Development, implementation, and impact of an automated early warning and response system for sepsis. J Hosp Med. 2015; 10(1):26–31. PMID: 25263548.
6. Subbe CP, Kruger M, Rutherford P, Gemmel L. Validation of a Modified Early Warning Score in medical admissions. QJM. 2001; 94(10):521–526. PMID: 11588210.
7. Liu V, Escobar GJ, Greene JD, Soule J, Whippy A, Angus DC, et al. Hospital deaths in patients with sepsis from 2 independent cohorts. JAMA. 2014; 312(1):90–92. PMID: 24838355.
8. Seymour CW, Liu VX, Iwashyna TJ, Brunkhorst FM, Rea TD, Scherag A, et al. Assessment of clinical criteria for sepsis: for the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016; 315(8):762–774. PMID: 26903335.
9. Levy MM, Evans LE, Rhodes A. The Surviving Sepsis Campaign bundle: 2018 update. Crit Care Med. 2018; 46(6):997–1000. PMID: 29767636.
10. Rhodes A, Evans LE, Alhazzani W, Levy MM, Antonelli M, Ferrer R, et al. Surviving Sepsis Campaign: international guidelines for management of sepsis and septic shock: 2016. Intensive Care Med. 2017; 43(3):304–377. PMID: 28101605.
11. Smith GB, Prytherch DR, Schmidt PE, Featherstone PI. Review and performance evaluation of aggregate weighted ‘track and trigger’ systems. Resuscitation. 2008; 77(2):170–179. PMID: 18249483.
12. Smith ME, Chiovaro JC, O’Neil M, Kansagara D, Quiñones AR, Freeman M, et al. Early warning system scores for clinical deterioration in hospitalized patients: a systematic review. Ann Am Thorac Soc. 2014; 11(9):1454–1465. PMID: 25296111.
13. Evans L, Rhodes A, Alhazzani W, Antonelli M, Coopersmith CM, French C, et al. Surviving Sepsis Campaign: international guidelines for management of sepsis and septic shock 2021. Crit Care Med. 2021; 49(11):e1063–e1143. PMID: 34605781.
14. Churpek MM, Snyder A, Han X, Sokol S, Pettit N, Howell MD, et al. Quick sepsis-related organ failure assessment, Systemic Inflammatory Response Syndrome, and Early Warning Scores for detecting clinical deterioration in infected patients outside the intensive care unit. Am J Respir Crit Care Med. 2017; 195(7):906–911. PMID: 27649072.
15. Churpek MM, Snyder A, Sokol S, Pettit NN, Edelson DP. Investigating the impact of different suspicion of infection criteria on the accuracy of quick sepsis-related organ failure assessment, Systemic Inflammatory Response Syndrome, and Early Warning Scores. Crit Care Med. 2017; 45(11):1805–1812. PMID: 28737573.
16. American College of Surgeons. Advanced Trauma Life Support Course for Physicians. Chicago, IL, USA: American College of Surgeons;1989.
17. Kelly CA, Upex A, Bateman DN. Comparison of consciousness level assessment in the poisoned patient using the alert/verbal/painful/unresponsive scale and the Glasgow Coma Scale. Ann Emerg Med. 2004; 44(2):108–113. PMID: 15278081.
18. Redfern OC, Smith GB, Prytherch DR, Meredith P, Inada-Kim M, Schmidt PE. A comparison of the quick sequential (sepsis-related) organ failure assessment score and the National Early Warning Score in non-ICU patients with/without infection. Crit Care Med. 2018; 46(12):1923–1933. PMID: 30130262.
19. Lo RS, Leung LY, Brabrand M, Yeung CY, Chan SY, Lam CC, et al. qSOFA is a poor predictor of short-term mortality in all patients: a systematic review of 410,000 patients. J Clin Med. 2019; 8(1):E61. PMID: 30626160.
20. Hillman K, Chen J, Cretikos M, Bellomo R, Brown D, Doig G, et al. Introduction of the medical emergency team (MET) system: a cluster-randomised controlled trial. Lancet. 2005; 365(9477):2091–2097. PMID: 15964445.
21. Song JU, Suh GY, Park HY, Lim SY, Han SG, Kang YR, et al. Early intervention on the outcomes in critically ill cancer patients admitted to intensive care units. Intensive Care Med. 2012; 38(9):1505–1513. PMID: 22592633.
22. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982; 143(1):29–36. PMID: 7063747.
23. Pencina MJ, D’Agostino RB Sr, D’Agostino RB Jr, Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 2008; 27(2):157–172. PMID: 17569110.
24. Churpek MM, Yuen TC, Huber MT, Park SY, Hall JB, Edelson DP. Predicting cardiac arrest on the wards: a nested case-control study. Chest. 2012; 141(5):1170–1176. PMID: 22052772.
25. Smith GB, Prytherch DR, Meredith P, Schmidt PE, Featherstone PI. The ability of the National Early Warning Score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death. Resuscitation. 2013; 84(4):465–470. PMID: 23295778.
26. Jones DA, DeVita MA, Bellomo R. Rapid-response teams. N Engl J Med. 2011; 365(2):139–146. PMID: 21751906.
27. Ludikhuize J, Smorenburg SM, de Rooij SE, de Jonge E. Identification of deteriorating patients on general wards; measurement of vital parameters and potential effectiveness of the Modified Early Warning Score. J Crit Care. 2012; 27(4):424.e7–424.13.
28. Clifton DA, Clifton L, Sandu DM, Smith GB, Tarassenko L, Vollam SA, et al. ‘Errors’ and omissions in paper-based early warning scores: the association with changes in vital signs--a database analysis. BMJ Open. 2015; 5(7):e007376.
29. Shappell C, Snyder A, Edelson DP, Churpek MM. American Heart Association’s Get With The Guidelines-Resuscitation Investigators. Predictors of in-hospital mortality after rapid response team calls in a 274 hospital nationwide sample. Crit Care Med. 2018; 46(7):1041–1048. PMID: 29293147.
30. Ahn JH, Jung YK, Lee JR, Oh YN, Oh DK, Huh JW, et al. Predictive powers of the Modified Early Warning Score and the National Early Warning Score in general ward patients who activated the medical emergency team. PLoS One. 2020; 15(5):e0233078. PMID: 32407344.
31. Serafim R, Gomes JA, Salluh J, Póvoa P. A comparison of the quick-SOFA and Systemic Inflammatory Response Syndrome criteria for the diagnosis of sepsis and prediction of mortality: a systematic review and meta-analysis. Chest. 2018; 153(3):646–655. PMID: 29289687.
32. Singer AJ, Ng J, Thode HC Jr, Spiegel R, Weingart S. Quick SOFA scores predict mortality in adult emergency department patients with and without suspected infection. Ann Emerg Med. 2017; 69(4):475–479. PMID: 28110990.
33. Liu VX, Lu Y, Carey KA, Gilbert ER, Afshar M, Akel M, et al. Comparison of Early Warning Scoring Systems for hospitalized patients with and without infection at risk for in-hospital mortality and transfer to the intensive care unit. JAMA Netw Open. 2020; 3(5):e205191. PMID: 32427324.
34. Jarvis SW, Kovacs C, Briggs J, Meredith P, Schmidt PE, Featherstone PI, et al. Are observation selection methods important when comparing early warning score performance? Resuscitation. 2015; 90:1–6. PMID: 25668311.

SUPPLEMENTARY MATERIALS

Supplementary Table 1

Modified Early Warning Score
jkms-37-e122-s001.doc

Supplementary Table 2

Baseline characteristics based on enrolled hospitals
jkms-37-e122-s002.doc

Supplementary Table 3

Prediction accuracy for in-hospital cardiac arrest according to different score thresholds
jkms-37-e122-s003.doc

Supplementary Table 4

Prediction accuracy for unexpected ICU transfer according to different score thresholds
jkms-37-e122-s004.doc
TOOLS
Similar articles