Verification of the Performance of the Panbio COVID-19 Ag Rapid Test Device for Implementation in the Clinical Laboratory

Hae-Sun Chung; Ji Su Chung; Yeo-Jin Lee; Seonwoo Lee; Juhyun Jeong; Min-Kyung So; Miae Lee

doi:10.12771/emj.2023.e2

ABSTRACT

Objectives:

The Panbio COVID-19 Ag Rapid Test Device (Panbio COVID-19 Ag, Abbott Rapid Diagnostics) is a lateral flow immunochromatographic assay targeting the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) nucleoprotein in nasopharyngeal specimens for the diagnosis of coronavirus disease 2019 (COVID-19). This study aimed to verify the performance of the Panbio COVID-19 Ag for implementation in clinical laboratories.

Methods:

Sixty nasopharyngeal swab specimens (30 positive and 30 negative) dipped in transport medium, and COVID-19 was confirmed using real-time RT-PCR using Allplex SARS-CoV-2 assay (Seegene), were tested using the Panbio COVID-19 Ag. Reproducibility was evaluated using positive and negative control materials. Sensitivity and specificity were calculated based on the results of real-time RT-PCR as the standard test method.

Results:

Reproducibility was confirmed by the consistent results of repeated tests of the quality control materials. The overall sensitivity and specificity of Panbio COVID-19 Ag were 50.0% and 100.0%, respectively. Panbio COVID-19 Ag demonstrated high sensitivity (88.2%) in analyzing the detection limit cycle threshold (Ct) value of 26.67 provided by the manufacturer as a positive criterion, and the sensitivity was 100.0% for the positive criterion of Ct values <25, although it was less sensitive for Ct ≥25.

Conclusion:

Considering the high sensitivity for positive samples with Ct values <25 and the rapid turnaround of results, Panbio COVID-19 Ag can be used in clinical laboratories to diagnose COVID-19 in limited settings.

Introduction

Laboratory diagnosis is important in promptly initiating appropriate management for individuals with coronavirus disease 2019 (COVID-19). Test methods for diagnosing COVID-19 should be reliable, affordable, accessible, and provide rapid results [1,2]. The standard reference method for laboratory diagnosis of COVID-19 is molecular testing, which is used to detect a specific gene of the causative pathogen—severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2)—for which real-time RT-PCR is most commonly used. Although this method yields the highest sensitivity and specificity among laboratory diagnostic tests for COVID-19, it has the disadvantages of requiring dedicated equipment, reagents, and skilled professionals. Moreover, it takes several hours to obtain results, making it difficult to initiate management promptly [1,3–5].

Antigen testing is a method that detects antigens composed of viral components, such as proteins. Antigen tests have been developed for the detection of SARS-CoV-2 and corresponding COVID-19 management. In particular, the rapid antigen detection test (RADT) can confirm results within 15–30 minutes in most cases, and the test method is simpler and easier than molecular tests; therefore, RADT can compensate for some obstacles encountered with molecular testing. The cost is also lower than that of molecular testing [1,4–6]. However, unlike genetic components, antigen components cannot be amplified; therefore, COVID-19 antigen testing requires at least 1,000 times more virus in a specimen than genetic testing; as such, sensitivity is low [1]. According to previous reports, most RADTs have reported a sensitivity of ≥80%–90% when the cutoff cycle threshold (Ct) was set as <25 and demonstrated low sensitivity for the subjects with the Ct was ≥25. In particular, it has been reported that a sensitivity <10% is observed when the subjects show Ct >30 [5–8].

The Panbio COVID-19 Ag Rapid Test Device (Panbio COVID-19 Ag, Abbott Rapid Diagnostics, Jena, Germany) is a diagnostic kit approved for use as an RADT for COVID-19. Panbio COVID-19 Ag is a lateral flow immunochromatographic assay targeting the SARS-CoV-2 nucleoprotein in nasopharyngeal specimens for the diagnosis of COVID-19. It reduces the time to read completion to 15 minutes compared to 30 minutes for other RADT previously used in our clinical laboratory, thus enabling faster reporting [9].

The present study aimed to verify the performance of the Panbio COVID-19 Ag for implementation in a clinical laboratory.

Methods

1. Specimens

Among the remnant nasopharyngeal swab specimens dipped in viral transport medium, 30 specimens each were confirmed positive and negative according to real-time RT-PCR testing. For positive cases, specimens collected within 1 month of evaluation and stored frozen (−70°C) were thawed immediately before testing; for negative cases, specimens collected and refrigerated the day before evaluation were used. Real-time RT-PCR testing was performed using the Allplex SARS-CoV-2 assay (Seegene, Seoul, Korea), and by applying the Ct of the RdRp/S gene as a standard, a similar number of specimens were selected for each Ct range. When collecting positive specimens, more than 30% of the positive specimens had a Ct value of 30 or more. This study was approved by the Institutional Review Board of Ewha Womans University Seoul Hospital.

2. Panbio COVID-19 Ag test

For the Panbio COVID-19 Ag test, 300 μL of specimen in transport medium was mixed with 300 μL of buffer, and five drops of the mixed solution was applied to the specimen well and reacted for 15 minutes. Among the samples in which the control line was positive, if both operators interpreted the result as positive, it was reported as positive and, if both operators interpreted the results as negative, it was reconfirmed 5 minutes later. If the operators’ opinions did not agree, the Panbio COVID-19 Ag test was retested with 300 μL of specimen without buffer and reported as the final result.

3. Verification of performance

1) Reproducibility

Using the quality control swab (positive and negative controls) enclosed in the Panbio COVID-19 Ag, the test was repeated 10 times to confirm the concordance of the results.

2) Comparison

The sensitivity and specificity of the Panbio COVID-19 Ag were calculated using real-time RT-PCR testing as a standard test method. For the Ct value, the Ct of the RdRp/S gene was applied.

Results

1. Reproducibility

Each quality control swab was tested 10 times, and the positive and negative controls were positive and negative, respectively, and all results were consistent.

2. Comparison test

The Panbio COVID-19 Ag results revealed that 15 of the 30 positive specimens were positive, and 15 were negative (Table 1). The sensitivity was 88.2% (95% CI 63.5%−98.2%), based on the detection limit Ct of 26.67 provided by the manufacturer (Table 2). Two specimens with a Ct ≤26.67, but negative for Panbio COVID-19 Ag, had Ct values of 25.89 and 26.62, respectively. The sensitivity was 100.0% (95% CI 78.0%−100.0%). for the positive criterion of Ct values <25. Specificity was 100%. Seven specimens were retested due to disagreement between the operators (Table 3).

Table 1.

Performance of the Panbio COVID-19 Ag Rapid Test Device for SARS-CoV-2 detection

Testing method		Real-time RT-PCR		Sensitivity (95% CI)	Specificity (95% CI)
Testing method		P	N	Sensitivity (95% CI)	Specificity (95% CI)
Panbio COVID-19 Ag	P	15	0	50.0% (31.3−68.7)	100.0% (88.3−100.0)
Panbio COVID-19 Ag	N	15	30	50.0% (31.3−68.7)	100.0% (88.3−100.0)

P, positive; N, negative.

Table 2.

Results of Panbio COVID-19 Ag Rapid Test Device and real-time RT-PCR by Ct range

Ct	Positive results (n)		Sensitivity (95% CI)
Ct	Real-time RT-PCR	Panbio COVID-19	Sensitivity (95% CI)
≤25	15	15	100.0% (78.0−100.0)
>25	15	0	0.0% (0.0−22.0)
≤26.67*	17	15	88.2% (63.5−98.2)
>26.67	13	0	0.0% (0.0−24.9)

Ct, cycle threshold.

^* Cutoff Ct provided by the manufacturer.

Table 3.

Retest results of Panbio COVID-19 Ag Rapid Test Device due to discrepancies in the initial results

Specimen	Results		Ct
Specimen	Initial (operater 1/operater 2)	Retest	Ct
P2	Positive/Negative	Positive	21.78
P18	Positive/Negative	Positive	22.37
P24	Positive/Negative	Negative	25.89
P20	Positive/Negative	Negative	26.62
P25	Positive/Negative	Negative	26.81
P16	Positive/Negative	Negative	27.18
P12	Positive/Negative	Negative	29.09

Ct, cycle threshold.

Discussion

In verifying the performance of the Panbio COVID-19 Ag, the sensitivity was low (50.0%) for all evaluated specimens compared with real-time RT-PCR. However, based on the detection limit Ct provided by the manufacturer, the sensitivity was high (88.2%) for positive specimens with a Ct value lower than the detection limit Ct. In particular, all specimens with a Ct <25 were considered to be positive (sensitivity, 100.0%).

Previous studies that most COVID-19 RADTs show a sensitivity of ≥80%−90% with cutoff Ct values <25, and a low sensitivity for Ct values ≥25 [5–8], which is consistent with the findings of the present study. In a meta-analysis of published studies, the pooled sensitivity of Panbio COVID-19 Ag was 71.8% (95% CI 65.4%−77.5%). According to the Ct value, the pooled sensitivity was 95.8% (95% CI 92.3%−97.8%) for Ct values <25 and 61.2% (95% CI 38.8%−79.7%) for Ct values >25 [7].

The sensitivity of the Panbio COVID-19 Ag has been reported in a very diverse range, and the target test group’s diversity is believed to be the most significant factor. A total of 39 studies evaluating Panbio COVID-19 Ag were included in the meta-analysis, and the sensitivity varied from 23.1% to 95.0% [7]. Among them, a multicenter study evaluating 958 patients (RT-PCR positivity rate, 37.5%) reported a sensitivity of 90.5% (95% CI 87.5%−93.6%); however, this study only included specimens collected from individuals 7 days from the onset of symptoms or from exposure to a confirmed case of COVID-19 [10]. Another study evaluated 293 symptomatic (55.1%) and 239 asymptomatic (44.9%) patients in the emergency room of a university hospital, with a sensitivity of 41.2% [11].

A meta-analysis revealed that the average sensitivity was 73.0% (95% CI 69.3%−76.4%) for symptomatic participants, which was higher than the average sensitivity of 54.7% for asymptomatic participants [8]. The performance evaluation results provided by the manufacturers were superior to those reported in published studies [9]. In the evaluation of specimens (140 positive, 445 negative) from symptomatic individuals, the sensitivity was 91.4% (95% CI 85.5%−95.5%). According to the Ct value, the sensitivity was 97.6% (95% CI, 93.2%−99.5%) for specimens with a Ct ≤30 and 94.1% (95% CI, 88.7%−97.4%) for those with a Ct ≤33. The results evaluated using specimens from asymptomatic individuals were less sensitive than those evaluated using specimens from symptomatic individuals. Of 483 specimens, 50 were RT-PCR positive, and the sensitivity for all positive specimens was 66.0% (95% CI 51.2%−78.8%). According to Ct value, the sensitivity was 93.8% (95% CI, 79.2%−99.2%) for specimens with a Ct ≤30 (n=32) and 80.0% (95% CI, 64.4%−90.9%) for those with a Ct ≤33 (n=40) [9]. Because clinical information was not collected in the present study, it was not possible to determine sensitivity according to whether patients were symptomatic or asymptomatic.

The lateral flow immunochromatographic assay interprets the result with the presence/absence of test and control lines. However, in some cases, it is difficult to read the lines by the naked eye when the intensity is low [12], and there may be differences among operators. Using a reader is known to improve these limitations [13], but Panbio COVID-19 Ag testing results are interpreted by naked eyes without a reader. Previous studies evaluating Panbio COVID-19 Ag have reported no differences between operators [14,15]. However, in this study, retests were performed for seven specimens due to discrepancies in the initial results interpreted by two operators. All specimens to be retested had Ct values in the range of 20−30 and, in particular, all specimens in the range of 25−30 (n=5) were retested. Because the detection limit Ct of Panbio COVID-19 Ag was 26.67, most of the retested specimens must have had virus concentrations near the detection limit. The cut-off, which is the criterion for distinguishing positive from negative in a qualitative test, can be both positive and negative through repeated tests. However, from a practical perspective, it is a disadvantage that a result is unclear or a retest is required. Although the COVID-19 RADT is simpler and easier to perform than molecular testing and does not require skilled operators, visual reads and interpretation may require some training and experience.

The World Health Organization (WHO) recommends that SARS-CoV-2 Ag RADTs that meet the minimum performance requirements of ≥80% sensitivity and ≥97% specificity compared with a nucleic acid amplification test reference assay can be used to diagnose COVID-19 in suspected cases of SARS-CoV-2 infection. According to the WHO guidelines, RADTs are less sensitive than nucleic acid amplification tests, particularly in asymptomatic populations; however, careful selection of cohorts for testing can mitigate this limitation. The WHO suggests that RADTs perform best in individuals with high viral loads and early in the course of infection, and will be most reliable in settings where the regional prevalence of SARS-CoV-2 infection is ≥5% [2]. The European Centre for Disease Prevention and Control (ECDC) agrees with the WHO minimum performance criteria of ≥80% sensitivity and ≥97% specificity but also advocates the use of higher performance tests (≥90% sensitivity and >98% specificity) [16]. In this study, Panbio COVID-19 Ag met the performance requirements of WHO and ECDC in specimens with Ct values <25.

Rather than evaluating the performance of Panbio COVID-19 Ag, this study aimed to verify its performance for implementation in a clinical laboratory; as such, there is a limitation in that the number of specimens was not sufficient. In addition, because the clinical information of individuals was not collected, the presence or absence of symptoms could not be determined.

In this study, the performance of the Panbio COVID-19 Ag was verified. Considering that Panbio COVID-19 Ag yielded high sensitivity at Ct values <25 and yielded results within 15 minutes, we believe that Panbio COVID-19 Ag can be useful in clinical laboratories. However, for RADTs to demonstrate proper performance, it is desirable to define a target group to be tested and to use it in limited situations, while considering the regional prevalence of COVID-19 and the accessibility of molecular tests and methods.

Acknowledgements

Not applicable.

Conflict of Interest

No potential conflict of interest relevant to this article was reported.

Author Contribution

Conceptualization: Chung HS, So MK, Lee M

Formal Analysis: Chung HS, Chung JS, Lee YJ, Lee S, Jeong J, So MK, Lee M

Investigation: Chung HS, Chung JS, Lee YJ, Lee S, Jeong J

Methodology: So MK, Lee M

Project Administration: Chung HS, So MK, Lee M

Writing – Original Draft: Chung HS, Chung JS, Lee YJ, Lee S, Jeong J

Writing – Review & Editing: Chung HS, Chung JS, Lee YJ, Lee S, Jeong J, So MK, Lee M

Ethics Approval and Consent to Participate

This study was approved by the Institutional Review Board of Ewha Womans University Seoul Hospital (Approval no. 2022-12-060).

References

1. Korean Society for Laboratory Medicine, Korea Disease Control and Prevention Agency. Guidelines for the laboratory diagnosis of COVID-19 in Korea [Internet]. Seoul (KR): Korean Society for Laboratory Medicine;c2021. [cited 2022 Dec 1]. Available from. https://www.kslm.org/rang_board/list.html?num=17518&start=225&code=covid19_press.

2. World Health Organization [WHO]. Antigen-detection in the diagnosis of SARS-CoV-2 infection [Internet]. Geneva (CH): WHO;c2021. [cited 2022 Dec 1]. Available from. https://www.who.int/publications/i/item/antigen-detection-in-the-diagnosis-of-sars-cov-2infection-using-rapid-immunoassays.

3. Hong KH, Lee SW, Kim TS, Huh HJ, Lee J, Kim SY, et al. Guidelines for laboratory diagnosis of coronavirus disease 2019 (COVID-19) in Korea. Ann Lab Med. 2020; 40(5):351–360. DOI: 10.3343/alm.2020.40.5.351. PMID: 32237288. PMCID: PMC7169629.

4. Lee K. Laboratory diagnosis of COVID-19 in Korea. Ewha Med. J 2021; 44(1):1–10. DOI: 10.12771/emj.2021.44.1.1.

5. Lee MK. Challenges of scaling up SARS-CoV-2 rapid antigen tests. Ann Clin Microbiol. 2021; 24(4):111–114. DOI: 10.5145/ACM.2021.24.4.1.

6. Mattiuzzi C, Henry BM, Lippi G. Making sense of rapid antigen testing in severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) diagnostics. Diagnosis. 2020; 8(1):27–31. DOI: 10.1515/dx-2020-0131. PMID: 33554523.

7. Brümmer LE, Katzenschlager S, Gaeddert M, Erdmann C, Schmitz S, Bota M, et al. Accuracy of novel antigen rapid diagnostics for SARS-CoV-2: a living systematic review and meta-analysis. PLoS Med. 2021; 18(8):e1003735. DOI: 10.1371/journal.pmed.1003735. PMID: 34383750. PMCID: PMC8389849.

8. Dinnes J, Sharma P, Berhane S, van Wyk SS, Nyaaba N, Domen J, et al. Rapid, point-of-care antigen tests for diagnosis of SARS-CoV-2 infection. Cochrane Database Syst Rev. 2022; 7(7):CD013705. DOI: 10.1002/14651858.CD013705.pub3. PMID: 35866452. PMCID: PMC9305720.

9. Abbott Rapid Diagnostics Jena GmbH. Panbio COVID-19 Ag Rapid Test Device Nasopharyngeal instructions for use [Internet]. Jena (DE): Abbott Rapid Diagnostics Jena GmbH;c2021. [cited 2022 Dec 1]. Available from. https://www.globalpointofcare.abbott/en/product-details/panbio-covid-19-ag-antigen-test.html.

10. Merino P, Guinea J, Muñoz-Gallego I, González-Donapetry P, Galán JC, Antona N, et al. Multicenter evaluation of the Panbio™ COVID-19 rapid antigen-detection test for the diagnosis of SARS-CoV-2 infection. Clin Microbiol Infect. 2021; 27(5):758–761. DOI: 10.1016/j.cmi.2021.02.001. PMID: 33601009. PMCID: PMC7884234.

11. Caruana G, Croxatto A, Kampouri E, Kritikos A, Opota O, Foerster M, et al. Implementing SARS-CoV-2 rapid antigen testing in the emergency ward of a Swiss University Hospital: the INCREASE Study. Microorganisms. 2021; 9(4):798. DOI: 10.3390/microorganisms9040798. PMID: 33920307. PMCID: PMC8069749.

12. Castrejón-Jiménez NS, García-Pérez BE, Reyes-Rodríguez NE, Vega-Sánchez V, Martínez-Juárez VM, Hernández-González JC. Challenges in the detection of SARS-CoV-2: evolution of the lateral flow immunoassay as a valuable tool for viral diagnosis. Biosensors. 2022; 12(9):728. DOI: 10.3390/bios12090728. PMID: 36140114. PMCID: PMC9496238.

13. Boehringer HR, O'Farrell BJ. Lateral flow assays in infectious disease diagnosis. Clin Chem. 2021; 68(1):52–58. DOI: 10.1093/clinchem/hvab194. PMID: 34969115. PMCID: PMC9383164.

14. Bulilete O, Lorente P, Leiva A, Carandell E, Oliver A, Rojo E, et al. Panbio™ rapid antigen test for SARS-CoV-2 has acceptable accuracy in symptomatic patients in primary health care. J Infect. 2021; 82(3):391–398. DOI: 10.1016/j.jinf.2021.02.014. PMID: 33592253. PMCID: PMC7881288.

15. Gremmels H, Winkel BMF, Schuurman R, Rosingh A, Rigter NAM, Rodriguez O, et al. Real-life validation of the Panbio™ COVID-19 antigen rapid test (Abbott) in community-dwelling subjects with symptoms of potential SARS-CoV-2 infection. EClinicalMedicine. 2021; 31:100677. DOI: 10.1016/j.eclinm.2020.100677. PMID: 33521610. PMCID: PMC7832943.

16. European Centre for Disease Prevention and Control [ECDC]. Options for the use of rapid antigen detection tests for COVID-19 in the EU/EEA – first update [Internet]. Stockholm (SE): ECDC;c2021. [cited 2022 Dec 1]. Available from. https://www.ecdc.europa.eu/en/publications-data/options-use-rapid-antigen-tests-covid-19-eueea-first-update.