Journal List > Lab Med Online > v.10(2) > 1145147

Evaluating the Diagnostic Performance of Two Rapid Influenza Diagnostic Tests

Abstract

Rapid influenza diagnostic test (RIDT) is widely used for the diagnosis of influenza owing to its simplicity and convenience of use. This study aimed to evaluate the performance of a new RIDT, SD Standard F influenza A/B FIA (SD Biosensor, Inc., Korea) (Standard F) and compare its performance with BD Veritor Flu A+B (Veritor), using the results of real-time reverse transcription PCR (rRT-PCR) analysis as the standard for reference. On comparing the results obtained from both the RIDTs and rRT-PCR qualitatively, it was found that the Veritor and Standard F assays have the sensitivity of 65.6% (21/32) and 71.9% (23/32), respectively, for the detection of influenza A with a specificity of 100.0% (68/68). Additionally, both the assays demonstrated a sensitivity of 66.7% (12/18) and specificity of 100.0% (68/68) for the detection of influenza B. The cutoff index (COI) value of the fluorescence color intensity from the Standard F assay, displayed on the device along with the qualitative results, indicated a negative correlation with the Ct value from rRT-PCR for both influenza A and B (P <0.001). The sensitivity of the new RIDT for the detection of influenza was comparable with that of the Veritor assay and the new RIDT could be used as a substitute for existing RIDTs by providing additional information to predict the approximate viral burden of influenza.

초록

인플루엔자 신속진단검사(rapid influenza diagnostic test, RIDT)는 사용의 편리함과 간편함으로 현재 인플루엔자 진단에 널리 사용되고 있다. 이 연구에서는 실시간 역전사 PCR (rRT-PCR)의 결과를 기준으로 새로운 국산 신속검사 키트인 SD Standard F influenza A/B FIA (SD Biosensor, Inc., Korea) (Standard F)와 BD Veri-tor Flu A+B (Veritor)의 성능을 비교 평가하였다. 본 연구에서 인플루엔자 A 검출에 대해 Veritor 및 Standard F 검사는 각각 65.6% 및 71.9%의 민감도와 100.0%의 특이도를 보였다. 인플루엔자 B 검출에 대해서는 두 RIDT 모두 66.7%의 민감도와 100.0%의 특이도를 보였다. Standard F assay에서 추가적으로 제시하는 형광 강도의 판정 기준치(cutoff index, COI)는 인플루엔자 A와 B에서 모두 rRT-PCR의 Ct 값과 음의 상관 관계를 보였다(P <0.001). Standard F는 기존의 신속검사 키트인 Veritor와 비교하여 유사한 민감도를 보였으며 추가적으로 인플루엔자 바이러스의 부하를 예측하는데 도움을 줄 수 있어 인플루엔자 바이러스 감염의 신속진단검사로 사용하는데 적합한 것으로 판단된다.

Influenza causes a broad range of illnesses, and its diagnosis based solely on clinical symptoms is difficult [1]. There are several diagnostic methods currently available including viral culture, molecular assays, serologic tests, and antigen detection tests. The sensitivity and specificity of the assays used for the detection of influenza virus vary depending on the type of method used [2, 3]. Molecular methods including rapid molecular assay and reverse transcription polymerase chain reaction (RT-PCR) are widely used as the standard methods for diagnosing influenza in clinical labo ratories. Molecular assays have high sensitivity and specificity for the detection of influenza virus; however, usually, appropriate laboratory settings and well-trained technicians are required to perform these tests. In contrast, rapid influenza diagnostic tests (RIDTs) are easier to handle and can detect influenza viral anti-gens in 10-15 minutes, making them valuable in the outpatient setting. Accordingly, RIDTs are widely used for the diagnosis of influenza as an initial step in clinical fields [2, 4]. Since results from RIDT methods are available before those from PCR or immunoassays, clinicians can make decisions on whether to start antiviral treatment based on these results [4]. However, the sensitivity of RIDTs is highly variable—ranging from 40-98% [2, 5, 6]—which can frequently lead to false negative results, thereby making it important to recognize the limitation of these methods while interpreting the results. In this study, we evaluated the diagnostic performance of a new RIDT, SD Standard F influenza A/B FIA (Standard F; SD Biosensor, Inc., Suwon, Korea), by comparing its performance with that of the commercially available RIDT, BD Veritor Flu A+B (Veritor; Becton Dickinson and Company Diagnostic, Sparks, MD, USA), relative to RT-PCR.
Both RIDTs involve immunochromatographic detection of nucleoprotein antigens of influenza A or B. The influenza viral antigens bind to the antiinfluenza antibodies conjugated to detector particles in the test strip [7]. The difference between the two RIDTs is that the Veritor uses colloidal-gold detector particles, whereas the Standard F uses fluorescent dyes as the detector. The results of both the assays are interpreted by a digital reading instrument. In case of the Veritor, only qualitative results such as positive or negative are displayed, whereas in Standard F, the qualitative results along with the fluorescence intensities as cutoff index (COI) values are displayed. According to the Standard F manufacturer’s instructions, test results of COI ≥1.00 are presented as positive and below 1.00 as negative for influenza.
We used 117 left-over, non-duplicated, nasopharyngeal samples from patients who visited the Severance hospital with a flu-like illness (fever >37.8°C and/or clinical symptoms of headache, cough, sore throat, and myalgia) from January 2018 to April 2018; the samples were from 12 adults and 105 children ranging from less than one to 17 years of age (median age, 2 years old). All the samples were analyzed by the AdvanSure RV real-time RT-PCR assay (rRT-PCR; LG Life Sciences, Seoul, Korea) and immediately preserved at -70°C after rRT-PCR. The thawing process was perfomed at room temperature (15-30°C) before analyzing the samples using the two RIDTs. The two RIDTs were performed simultaneously and all the results obtained were compared with those from the rRT-PCR analysis. We interpreted samples with a cycle threshold (Ct) value <25.00 as a positive result according to the manufacturer’s instructions. This study has been approved by the Institutional Review Board (IRB) of Yonsei University College of Medicine (IRB 1-2017-0093).
Sensitivity, specificity, positive predictive value, and negative predictive values with 95% confidence intervals (CIs) were calculated using standard formulas. Statistical analysis was performed using Microsoft Excel (Microsoft Corporation, WA, USA) and Analyse-it software, Method Evaluation edition version 5.11 (Analyze-it Software LTD, City West Business Park, Leeds, UK). Pearson correlation coefficient was calculated to evaluate the correlation between the Ct values from rRT-PCR and COI values from the Stan-dard F. P values <0.05 were considered to indicate statistical sig-nificance.
Sensitivity and specificity of the two RIDTs for the detection of influenza are shown in Table 1. After rRT-PCR analysis for influ-enza A and B (A/B), 31 of the 117 specimens were found to be A positive/B negative, 17 were A negative/B positive, only one was A positive/B positive, and the rest 68 were A negative/B negative. Using rRT-PCR as the testing standard, the Veritor and Standard F assays were found to demonstrate a sensitivity of 65.6% (21/32) and 71.9% (23/32), respectively, for the detection of influenza A with a detection specificity of 100.0% (68/68). Additionally, both the assays demonstrated a sensitivity of 66.7% (12/18) and speci city of 100.0% (68/68) for the detection of influenza B. Fig. 1 shows comparison of Ct values of the influenza-positive specimens. A total of 49 samples were found to be positive by rRT-PCR, and almost all the samples with low Ct values, below 14.4 for influenza A and 13.3 for influenza B, were detected by both the assays.
In our study, Standard F showed a diagnostic performance com-parable with that of Veritor in detecting influenza A infection with a higher sensitivity. Two out of the 31 rRT-PCR influenza A positive samples (Ct value of 9.79 and 20.31) showed discordant results between the two RIDTs, positive by Standard F and negative by Veritor. Those cases were clinically diagnosed with influenza A infection and treated with medication, which was in line with the results obtained by the Standard F method. Our result is compara-ble with a previous report that reveals improved sensitivity and a lowered limit of detection (LOD) of fluorescent immunochromatography-based assays compared to that of colloidal gold-based rapid diagnostic kits [8]. However, the sensitivity of both the RIDT as-says was lower than the value suggested by the manufacturer. The differences in sensitivity could be due to the nature of the samples that we used. The study was conducted using samples that were preserved at -70°C after RT-PCR analysis. It is possible that degradation of the antigens during the thawing process could have resulted in the unexpectedly lower sensitivity values [9]. It is possible that the RIDT assays could not detect certain influenza-positive samples confirmed by PCR due to the presence of an altered epitope due to minor changes in the protein structure that was not detectable by the kit, and this could have led to differences between the sensitivity reported in our study and that suggested by the manufacturer [10].
The Standard F test displayed not only qualitative results but also quantitative COI values. The COI values displayed a moderate negative correlation with Ct values of the influenza-positive specimens (influenza A: R=-0.77, P <0.001; influenza B: R=-0.88, P <0.001; Fig. 2). Kim et al. [11] and Ryu et al. [12] studied dilution tests for fluorescent lateral flow immunoassays and described a correlation between the COI values and concentration of the target epitope such as rotavirus and antibody to hepatitis B surface antigen. Our results also demonstrate that the COI values might reflect the influenza viral load and can help estimate the relative amount of influenza antigen. Therefore, the COI values can provide additional information and guidance to a clinician analyzing the negative results. If the result of a symptomatic patient is negative with a COI value just below the cut-off level, a clinician can consider the possibility of early infection with low viral load; in such circumstances, a repeat test can be conducted within a short time-frame, or other viral assays such as RT-PCR can be given immediate consideration.
Consistent with previous studies, the two RIDTs showed a slightly higher overall sensitivity for influenza A (65.6%-71.9 %) than for in-fluenza B (66.7%) [5, 13-15] and the sensitivity and specificity of both the RIDTs were higher than the pooled values of sensitivity and specificity of 53.2%-62.3% and 98.2%-99.8%, respectively [2, 5]. An earlier study of the Veritor test revealed a sensitivity of 73.0%-93.8% for influenza A and 55.6%-94.2% for influenza B, and an overall specificity greater than 95% [2, 7, 14, 16-19]. These values are slightly higher than those reported in our current study. These wide ranges of sensitivity can be explained by factors including specimen type, assay kit lot, the phase of infection (related to viral load and shedding), and differences in the targeted study population; the sensitivity of the tests has been found to be higher for samples obtained from children [9, 18, 20].
Our study has some limitations. First, we conducted this study with a small number of samples. In addition, we did not use fresh samples, and this might have had a negative influence on the sensitivity of the assays.
In conclusion, the Standard F method shows a comparable re-sult for diagnosing influenza infection, is as sensitive as other RIDTs, and provides additional semi-quantitative information that can aid in diagnosis.

Acknowledgments

We are thankful to SD Biosensor (Suwon, Korea) for providing us with the test reagents and automated reader.

Notes

Conflicts of Interest

None declared.

REFERENCES

1. Nicholson KG, Wood JM, Zambon M. 2003; Influenza. Lancet. 362:1733–45. DOI: 10.1016/S0140-6736(03)14854-4. PMID: 14643124. PMCID: PMC7112395.
crossref
2. Merckx J, Wali R, Schiller I, Caya C, Gore GC, Chartrand C, et al. 2017; Diagnostic accuracy of novel and traditional rapid tests for influenza infec-tion compared with reverse transcriptase polymerase chain reaction: A systematic review and metaanalysis. Ann Intern Med. 167:394–409. DOI: 10.7326/M17-0848. PMID: 28869986.
3. Centers for disease control and prevention (CDC). Information for clinicians on influenza virus testing. https://www.cdc.gov/flu/profes-sionals/diagnosis/index.htm. Updated on Feb. 2018.
4. Tsushima Y, Uno N, Sasaki D, Morinaga Y, Hasegawa H, Yanagihara K. 2015; Quantitative RT-PCR evaluation of a rapid influenza antigen test for efficient diagnosis of influenza virus infection. J Virol Methods. 212:76–9. DOI: 10.1016/j.jviromet.2014.10.019. PMID: 25449113.
crossref
5. Chartrand C, Leeflang MM, Minion J, Brewer T, Pai M. 2012; Accuracy of rapid influenza diagnostic tests: a meta-analysis. Ann Intern Med. 156:500–11. DOI: 10.7326/0003-4819-156-7-201204030-00403. PMID: 22371850.
6. World Health Organization. WHO recommendations on the use of rapid testing for influenza diagnosis. https://www.who.int/influenza/resources/documents/rapid_testing/en/. Updated on Jul. 2005.
7. Park S, Lee M, Chung HS. 2015; Evaluation of the performance of a new chro-matographic assay BD Veritor™ System for rapid detection of influenza A & B. Ann Clin Microbiol. 18:27–32. DOI: 10.5145/ACM.2015.18.1.27.
8. Yeo SJ, Bao DT, Seo GE, Bui CT, Kim DTH, Anh NTV, et al. 2017; Improvement of a rapid diagnostic application of monoclonal antibodies against avian influenza H7 subtype virus using Europium nanoparticles. Sci Rep. 7:7933. DOI: 10.1038/s41598-017-08328-9. PMID: 28801679. PMCID: PMC5554140.
crossref
9. Peaper DR and Landry ML. 2014; Rapid diagnosis of influenza: state of the art. Clin Lab Med. 34:365–85. DOI: 10.1016/j.cll.2014.02.009. PMID: 24856533. PMCID: PMC7172071.
10. Webster RG, Bean WJ, Gorman OT, Chambers TM, Kawaoka Y. 1992; Evo-lution and ecology of influenza A viruses. Microbiol Mol Biol Rev. 56:152–79. DOI: 10.1128/MMBR.56.1.152-179.1992. PMID: 1579108. PMCID: PMC372859.
crossref
11. Kim JS, Lee SK, Ko DH, Hyun J, Kim HS. 2019; Performance evaluation of the automated fluorescent immunoassay system rotavirus assay in clinical samples. Ann Lab Med. 39:50–7. DOI: 10.3343/alm.2019.39.1.50. PMID: 30215230. PMCID: PMC6143470.
crossref
12. Ryu JH, Kwon M, Moon JD, Hwang MW, Lee JM, Park KH, et al. 2018; Development of a rapid automated fluorescent lateral flow immunoassay to detect hepatitis B surface antigen (HBsAg), antibody to HBsAg, and antibody to hepatitis C. Ann Lab Med. 38:578–84. DOI: 10.3343/alm.2018.38.6.578. PMID: 30027702. PMCID: PMC6056386.
crossref
13. Yoo Y, Sohn JW, Park DW, Kim JY, Shin HK, Lee Y, et al. 2007; Clinical evaluation of the SD Bioline influenza virus antigen test for rapid detection of influenza viruses A and B in children and adults during the influenza season. Clin Vaccine Immunol. 14:1050–2. DOI: 10.1128/CVI.00465-06. PMID: 17567765. PMCID: PMC2044486.
crossref
14. Ryu SW, Suh IB, Ryu SM, Shin KS, Kim HS, Kim J, et al. 2018; Comparison of three rapid influenza diagnostic tests with digital readout systems and one conventional rapid influenza diagnostic test. J Clin Lab Anal. 32:e22234. DOI: 10.1002/jcla.22234. PMID: 28407318. PMCID: PMC6817280.
crossref
15. Self WH, McNaughton CD, Grijalva CG, Zhu Y, Chappell JD, Williams JV, et al. 2012; Diagnostic performance of the BinaxNow Influenza A&B rapid antigen test in ED patients. Am J Emerg Med. 30:1955–61. DOI: 10.1016/j.ajem.2012.04.018. PMID: 22795995.
16. Mese S, Akan H, Badur S, Uyanik A. Istanbul Rapid Test Study Group. 2016; Analytical performance of the BD Veritor™ System for rapid detection of influenza virus A and B in a primary healthcare setting. BMC Infect Dis. 16:481. DOI: 10.1186/s12879-016-1811-9. PMID: 27612949. PMCID: PMC5016879.
crossref
17. Hassan F, Nguyen A, Formanek A, Bell JJ, Selvarangan R. 2014; Comparison of the BD Veritor™ System for Flu A+B with the Alere BinaxNOW® In-fluenza A&B card for detection of influenza A and B viruses in respiratory specimens from pediatric patients. J Clin Microbiol. 52:906–10. DOI: 10.1128/JCM.02484-13. PMID: 24391198. PMCID: PMC3957768.
18. Ryu SW, Lee JH, Kim J, Jang MA, Nam JH, Byoun MS, et al. 2016; Comparison of two new generation influenza rapid diagnostic tests with instrument-based digital readout systems for influenza virus detection. Br J Biomed Sci. 73:115–20. DOI: 10.1080/09674845.2016.1189026. PMID: 27327199.
crossref
19. Dunn J, Obuekwe J, Baun T, Rogers J, Patel T, Snow L. 2014; Prompt detec-tion of influenza A and B viruses using the BD Veritor™ System Flu A+B, Quidel® Sofia® Influenza A+B FIA, and Alere BinaxNOW® Influ-enza A&B compared to real-time reverse transcription-polymerase chain reaction (RT-PCR). Diagn Microbiol Infect Dis. 79:10–3. DOI: 10.1016/j.diagmicrobio.2014.01.018. PMID: 24582581.
20. Selove W and Rao LV. 2016; Performance of rapid SOFIA Influenza A+B test compared to Luminex x-TAG respiratory viral panel assay in the diagnosis of influenza A, B, and subtype H3. J Investig Med. 64:905–7. DOI: 10.1136/jim-2016-000055. PMID: 26911275. PMCID: PMC4819670.

Fig. 1
Comparison of cycle threshold (Ct) values of the influenza-positive specimens. (A), influenza A: N=32 and (B), influenza B: N=18. Specimens with negative RIDT results are indicated by an empty box. The maximum Ct value with positive RIDT result was 20.31 and 13.26 for influenza A and B, respectively.
Abbreviations: rRT-PCR, real-time reverse transcription polymerase chain reaction; RIDT, rapid influenza diagnostic test; Veritor, BD Veritor assay; Standard F, SD Standard F assay.
LMO-10-160-f1.tif
Fig. 2
Correlation of cutoff index (COI) values obtained from SD Standard F assay with cycle threshold (Ct) values of influenza-positive specimens. (A), influenza A and (B), influenza B.
Abbreviation: rRT-PCR, real-time reverse transcription polymerase chain reaction.
LMO-10-160-f2.tif
Table 1
Performance of two rapid influenza diagnostic tests for the detection of influenza in comparison with rRT-PCR
Rapid test rRT-PCR test % (95% CI)


A B A+B N Sensitivity Specificity
Veritor A 20 0 0 0 65.6 (46.8-81.4) 100.0 (94.7-100.0)
B 0 11 0 0 66.7 (40.1-86.1) 100.0 (94.7-100.0)
A+B 0 0 1 0
N 11 6 0 68
Standard F A 22 0 0 0 71.9 (53.3-86.3) 100.0 (94.7-100.0)
B 0 11 0 0 66.7 (40.1-86.1) 100.0 (94.7-100.0)
A+B 0 0 1 0
N 9 6 0 68

Abbreviations: rRT-PCR, real-time reverse transcription polymerase chain reaction; A, influenza A; B, influenza B; A+B, influenza A, B co-infection; N, negative; Veritor, BD Veritor assay; Standard F, SD Standard F assay.

TOOLS
Similar articles