Real-World Accuracy of a SARS-CoV-2 Rapid Diagnostic Tests in the Republic of Korea

Oh Joo Kweon; Hye Ryoun Kim; Mi-Kyung Lee; Yong Kwan Lim

doi:10.3346/jkms.2022.37.e310

Abstract

Antigen rapid diagnostic tests (RDTs) became the most important tool for the diagnosis of the coronavirus disease 2019 (COVID-19), however there have been very few evaluations of the accuracy of the RDTs in actual use. In this study, we investigated the performance accuracy of the RDT, the STANDARD Q COVID-19 Ag (STANDARD Q), in the Republic of Korea. We collected a total of 5,792 results that underwent both RDT and reverse transcription polymerase chain reaction simultaneously, and overall sensitivity and specificity of the STANDARD Q were 57.6% and 99.9%, respectively. With binomial logistic regression analysis, we estimated that about half of the COVID-19 patients with a cycle threshold value of 25 for E and RdRP were RDT-negative. These results suggest that the clinical sensitivity of RDTs against severe acute respiratory syndrome coronavirus 2 is considerably low in a real-world setting, and we recommend that limitations of RDTs should be considered when setting up COVID-19 test strategies.

About two years have passed since a novel strain of coronavirus, severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), has spread rapidly worldwide.1 During this outbreak, antigen rapid diagnostic tests (RDTs) became one of the most important options for detection of the coronavirus disease 2019 (COVID-19).2 In particular, RDTs have a major advantage in terms of short turnaround times and low running costs in several countries that lack resources. Many RDTs have been evaluated for performance in various studies; however, there have been very few evaluations of the accuracy of the test in actual use.3 Therefore, we investigated the performance accuracy of the RDT, which is used globally.

The STANDARD Q COVID-19 Ag (STANDARD Q; SD Biosensor, Inc., Cheongju, Korea) is a lateral flow-based RDT that was developed during the early SARS-CoV-2 pandemic. This kit was introduced in December 2020 at Chung-Ang University Hospital (CAUH) and has been used as a screening tool for COVID-19 patients visiting the emergency room. Between December 2020 and April 2022, a total of 5,792 patients were tested with STANDARD Q, and 5,496 patients also underwent reverse transcription polymerase chain reaction (RT-PCR) testing using nasopharyngeal/oropharyngeal swab specimens. The routine SARS-CoV-2 RT-PCR tests were performed with Allplex™ 2019-nCoV Assay (Seegene, Busan, Korea) or Real-Q Direct SARS-CoV2 Detection Kit (BioSewoom, Seoul, Korea), and we assumed that the cycle threshold (Ct) values of two PCR assays were equivalent. In this study, we retrospectively reviewed these test results and estimated the real-world sensitivity and specificity of STANDARD Q compared to RT-PCR results. In addition, binomial logistic regression analysis was used to estimate the limit of detection (LOD) for STANDARD Q. This analysis was used to predict the functions between the RDT results and the Ct value of each target gene (E and RdRp), and to estimate the Ct values at which 50% and 95% detection rates were achieved (LOD_95% and LOD_50%).

Among the patients who underwent both RDT and RT-PCR (n = 5,496), 418, 36, and 5,042 showed positive, indeterminate, and negative RT-PCR results, respectively. Since it was impossible to confirm whether patients with indeterminate results were infected with SARS-CoV-2, we excluded these patients from this study. Among the RT-PCR-positive patients (n = 418), 239 and 179 showed positive and negative results in the STANDARD Q assay, respectively. In addition, three cases showed RT-PCR-negative and RDT-positive results. Overall sensitivity and specificity of the STANDARD Q were 57.2% (95% confidence interval [CI], 52.4–61.8) and 99.9% (95% CI, 99.8–100), respectively. When considering the overall prevalence of COVID-19 in this study (about 8%), the positive and negative predictive values (PPV and NPV) were 98.8% (95% CI, 96.4–99.6%) and 96.6% (95% CI, 96.0–97.0%), respectively. In the binomial logistic regression analysis, the LOD_95% and LOD_50% values were 17.2 and 25.0 for E, and 17.8 and 25.7 for RdRp, respectively (Fig. 1).

According to the interim guidance of the World Health Organization (WHO), SARS-CoV-2 RDTs should meet the minimum performance requirements of ≥ 80% sensitivity and ≥ 97% specificity,4 and most RDT manufacturers claim that the sensitivities and specificities of RDTs meet the performance criteria presented in this guideline. In addition, there have been many reports that the sensitivity of RDTs is satisfactory for the diagnosis and management of COVID-19.5 6 However, when conducting long-term studies in clinical settings, there are many reports that the clinical performance of RDTs is significantly inferior to that of the RT-PCR assays.7 8 Similar to these results, we estimated that the sensitivity of STANDARD Q was 57.6%, which is lower than the sensitivity criteria presented in the WHO guidelines and the manufacturer’s specifications.

In this study, the LOD_95% for RdRP was 17.8, and this result were much lower than the manufacturer-claimed RdRP Ct values of 23.37.9 In addition, the LOD_50% values of E and RdRp were approximately 25 Ct values, which means that half of the COVID-19 patients with a Ct value of 25 were RDT-negative. Although the correlation between the infectivity of SARS-CoV-2 and the Ct values of COVID-19 RT-PCR assays is not fully known, Rhee et al.10 reported that approximately 80% of specimens were positive in SARS-CoV-2 culture when the Ct value of RT-PCR was 25. Considering these results together, we can assume that a significant number of patients with SARS-CoV-2 infectivity were falsely classified as SARS-CoV-2 negative when RDTs were widely used to diagnose COVID-19.11 Recently, an explosive number of COVID-19 confirmed cases (approximately 100,000 to 600,000 patients) have occurred daily in the Republic of Korea, and we hypothesize that the introduction of RDTs as a COVID-19 confirmatory test may have influenced these outbreaks because the test failed to detect a large number of infectious patients who were COVID-19 positive.

There were several limitations in this study. First, due to the nature of retrospective study, a selection bias could occur, such as the inclusion of more patients with severe symptoms. Second, we could not analyze the change of sensitivity and specificity according to the symptoms, because we could not conduct chart review for enrolled patients.

In conclusion, the clinical sensitivity of RDTs for the detection of SARS-CoV-2 is considerably low in a real-world setting, and we recommend that the limitations of RDTs should be considered when setting up COVID-19 test strategies.

Ethics statement

This study was approved by the Institutional Review Board of CAUH (IRB No. 2205-006-19417), and the need for informed consent was waived by the board.

ACKNOWLEDGMENTS

We thank the Diagnostic Immunology Department of the Chung-Ang University Hospital for their contribution to this study for a long time.

Notes

Funding: This work was supported by a National Research Foundation of Korea (NRF) grant funded by the Korean government (MSIT) (No.2020R1A5A1018052).

Disclosure: The authors have no potential conflicts of interest to disclose.

Author Contributions:

Conceptualization: Kim HR, Lim YK.
Data curation: Kim HR.
Investigation: Kweon OJ.
Methodology: Lee MK.
Resources: Lee MK.
Writing - original draft: Kweon OJ, Lim YK.
Writing - review & editing: Kweon OJ, Lee MK, Kim HR, Lim YK.

References

1. Kweon OJ, Lim YK, Kim HR, Choi Y, Kim MC, Choi SH, et al. Evaluation of rapid SARS-CoV-2 antigen tests, AFIAS COVID-19 Ag and ichroma COVID-19 Ag, with serial nasopharyngeal specimens from COVID-19 patients. PLoS One. 2021; 16(4):e0249972. PMID: 33831118.

2. Lim YK, Kweon OJ, Kim HR, Kim TH, Lee MK. Field evaluation of seven SARS-COV-2 antigen assays. J Infect Public Health. 2022; 15(2):199–202. PMID: 34991002.

3. Lee J, Kim SY, Huh HJ, Kim N, Sung H, Lee H, et al. Clinical performance of the standard Q COVID-19 rapid antigen test and simulation of its real-world application in Korea. Ann Lab Med. 2021; 41(6):588–592. PMID: 34108286.

4. World Health Organization. Antigen-detection in the diagnosis of SARS-CoV-2 infection using rapid immunoassays 2020. Updated 2021. Accessed May 1, 2022. https://www.who.int/publications-detail-redirect/antigen-detection-in-the-diagnosis-of-sars-cov-2infection-using-rapid-immunoassays .

5. Rahman MM, Hoque AF, Karim Y, Kawser Z, Siddik AB, Sumiya MK, et al. Clinical evaluation of SARS-CoV-2 antigen-based rapid diagnostic test kit for detection of COVID-19 cases in Bangladesh. Heliyon (Lond). 2021; 7(11):e08455.

6. Corman VM, Haage VC, Bleicker T, Schmidt ML, Muhlemann B, Zuchowski M, et al. Comparison of seven commercial SARS-CoV-2 rapid point-of-care antigen tests: a single-centre laboratory evaluation study. Lancet Microbe. 2021; 2(7):e311–e319. PMID: 33846704.

7. Jegerlehner S, Suter-Riniker F, Jent P, Bittel P, Nagler M. Diagnostic accuracy of a SARS-CoV-2 rapid antigen test in real-life clinical settings. Int J Infect Dis. 2021; 109:118–122. PMID: 34242764.

8. Jeewandara C, Guruge D, Pushpakumara PD, Madhusanka D, Jayadas TT, Chaturanga IP, et al. Sensitivity and specificity of two WHO approved SARS-CoV2 antigen assays in detecting patients with SARS-CoV2 infection. BMC Infect Dis. 2022; 22(1):276. PMID: 35317731.

9. Lee J, Kim SY, Huh HJ, Kim N, Sung H, Lee H, et al. Clinical performance of the Standard Q COVID-19 rapid antigen test and simulation of its real-world application in Korea. Ann Lab Med. 2021; 41(6):588–592. PMID: 34108286.

10. Rhee C, Kanjilal S, Baker M, Klompas M. Duration of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infectivity: when is it safe to discontinue isolation? Clin Infect Dis. 2021; 72(8):1467–1474. PMID: 33029620.

11. Kweon OJ, Lee JH, Choi YS, Kim BS, Lim YK, Lee MK, et al. Positivity of rapid antigen testing for SARS-CoV-2 with serial followed-up nasopharyngeal swabs in hospitalized patients due to COVID-19. J Korean Med Sci. 2022; 37(21):e168. PMID: 35638195.