Journal List > Korean J Lab Med > v.26(4) > 1011333

TOOLS
Kim, Oh, Cha, Choi, An, and Lee: Quality Evaluation of the Performance Study of Diagnostic Tests Using STARD Checklist and Meta-Analysis for the Pooled Sensitivity and Specificity of Third Generation Anti-HCV EIA Tests
Original Article

Korean J Leg Med 2006;26(4):307-315.

Published online: 10 February 2006

DOI: https://doi.org/10.3343/kjlm.2006.26.4.307

Quality Evaluation of the Performance Study of Diagnostic Tests Using STARD Checklist and Meta-Analysis for the Pooled Sensitivity and Specificity of Third Generation Anti-HCV EIA Tests

Sollip Kim, M.D.1, Heung-Bum Oh, M.D.1, Chung-Hwan Cha, M.D.1, Sung-Eun Choi, M.A.1, Hong-yup An, Ph.D.2, Kwan Jeh Lee, Ph.D.2

1Department of Laboratory Medicine, Asan Medical Center and University of Ulsan College of Medicine, Seoul, Korea

2Department of Statistics, Dongguk University, Seoul, Korea

Received 18 April 2006       Revised 15 May 2006       Accepted 17 May 2006

Copyright © 2006 The Korean Society for Legal Medicine

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background

The third generation anti-hepatitis C virus (HCV) enzyme immunoassay (EIA) is now in use for screening HCV infection. The aim of this study was to pool the data on the sensitivity and specificity of third generation anti-HCV EIA tests after evaluating the quality of the studies using Standards for Reporting of Diagnostic Accuracy studies (STARD) checklist.

Methods

We searched MEDLINE and PubMed databases using keywords about the accuracy of diagnostic tests for HCV infections. Methodological quality was assessed by two persons with a modified STARD checklist. A heterogeneity test was performed, and in case heterogeneity was present, a sub-group analysis was done. Fixed-effects model was used to obtain pool sensitivity and specificity with 95% confidence intervals (CI).

Results

A total of 41 studies from 16 papers were selected. The quality score ranged from 6 to 13 (median 10.5); Inter-observer agreement was 93.62% (k=0.69); and 41 studies revealed heterogeneity. We performed a sub-group analysis with only 28 studies from 13 papers that were evaluated to be of high quality. A subgroup using polymerase chain reaction as the reference test revealed homogeneity and was calculated the pooled sensitivity and specificity of 99.92% (CI 99.77–100.07%) and 99.66% (CI 99.45–99.86%) respectively. Studies on test kits with an increased reactivity to the core region also showed homogeneity in sensitivity and the pooled sensitivity was 99.78% (CI 99.53–100.03%).

Conclusions

For the first time in Korea, the diagnostic accuracy of test kits was evaluated by meta-analysis using STARD checklist. The methodology shown in this study should help extending laboratory medicine to an evidence-based medicine.

Keywords: STARD, Meta-analysis, Third generation anti-HCV EIA

References

1. Shepard CW, Finelli L, Alter MJ. Global epidemiology of hepatitis C virus infection. Lancet Infect Dis. 2005; 5:558–67.
crossref
2. Suh DJ, Jeong SH. Current status of hepatitis C virus infection in Korea. Intervirology. 2006; 49:70–5.
crossref
3. Oh HB, Hwang YS, Cho YJ, Kim DS, Kim SI. Experience of anti-HCV antibody immunoblot test in Korean blood donors. Korean J Blood Transfus. 1997; 8:1–8.
4. Wong T, Lee SS. Hepatitis C: a review for primary care physicians. CMAJ. 2006; 174:649–59.
crossref
5. Colin C, Lanoir D, Touzet S, Meyaud-Kraemer L, Bailly F, Trepo C. Sensitivity and specificity of third-generation hepatitis C virus antibody detection assays: an analysis of the literature. J Viral Hepat. 2001; 8:87–95.
crossref
6. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, et al. Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. Standards for Reporting of Diagnostic Accuracy. Clin Chem. 2003; 49:1–6.
7. Re V, Gallego S, Trevino E, Barbas G, Dominguez C, Elbarcha O, et al. Evaluation of five screening tests licensed in Argentina for detection of hepatitis C virus antibodies. Mem Inst Oswaldo Cruz. 2005; 100:303–7.
8. Berger A, Doerr HW, Preiser W, Weber B. Lack of correlation between different hepatitis C virus screening and confirmatory assays. J Virol Methods. 1996; 59:141–6.
crossref
9. Courouce AM, Barin F, Botte C, Lunel F, Maisonneuve P, Maniez M, et al. A comparative evaluation of the sensitivity of seven anti-hepatitis C virus screening tests. Vox Sang. 1995; 69:213–6.
10. Hennig H, Schlenke P, Kirchner H, Bauer I, Schulte-Kellinghaus B, Bludau H. Evaluation of newly developed microparticle enzyme immunoassays for the detection of HCV antibodies. J Virol Methods. 2000; 84:181–90.
crossref
11. Jonas G, Pelzer C, Beckert C, Hausmann M, Kapprell HP. Performance characteristics of the ARCHITECT anti-HCV assay. J Clin Virol. 2005; 34:97–103.
crossref
12. Zachary P, Ullmann M, Djeddi S, Wendling MJ, Schvoerer E, Stoll-Keller F, et al. Evaluation of two commercial enzyme immunoassays for diagnosis of hepatitis C in the conditions of a virology laboratory. Pathol Biol (Paris). 2004; 52:511–6.
13. Lee SR, Wood CL, Lane MJ, Francis B, Gust C, Higgs CM, et al. Increased detection of hepatitis C virus infection in commercial plasma donors by a third-generation screening assay. Transfusion. 1995; 35:845–9.
crossref
14. Vrielink H, Zaaijer HL, Reesink HW, van der Poel CL, Cuypers HT, Lelie PN. Sensitivity and specificity of three third-generation anti-hepatitis C virus ELISAs. Vox Sang. 1995; 69:14–7.
crossref
15. Abdel-Hamid M, El-Daly M, El-Kafrawy S, Mikhail N, Strickland GT, Fix AD. Comparison of second- and third-generation enzyme immunoassays for detecting antibodies to hepatitis C virus. J Clin Microbiol. 2002; 40:1656–9.
crossref
16. Ismail N, Fish GE, Smith MB. Laboratory evaluation of a fully automated chemiluminescence immunoassay for rapid detection of HBsAg, antibodies to HBsAg, and antibodies to hepatitis C virus. J Clin Microbiol. 2004; 42:610–7.
crossref
17. Huber KR, Sebesta C, Bauer K. Detection of common hepatitis C virus subtypes with a third-generation enzyme immunoassay. Hepatology. 1996; 24:471–3.
18. Prince AM, Scheffel JW, Moore B. A search for hepatitis C virus polymerase chain reaction-positive but seronegative subjects among blood donors with elevated alanine aminotransferase. Transfusion. 1997; 37:211–4.
crossref
19. Kodama T, Ichiyama S, Sato K, Nada T, Nakashima N. Evaluation of a membrane filter assay system, Ortho HCV Ab Quick Pack, for detection of anti-hepatitis C virus antibody. J Clin Microbiol. 1998; 36:1439–40.
20. Stuyver L, Claeys H, Wyseur A, Van Arnhem W, De Beenhouwer H, Uytendaele S, et al. Hepatitis C virus in a hemodialysis unit: molecular evidence for nosocomial transmission. Kidney Int. 1996; 49:889–95.
crossref
21. Lavanchy D, Steinmann J, Moritz A, Frei PC. Evaluation of a new automated third-generation anti-HCV enzyme immunoassay. J Clin Lab Anal. 1996; 10:269–76.
crossref
22. Marcellin P, Martinot-Peignoux M, Gabriel F, Branger M, Degott C, Elias A, et al. Chronic non-B, non-C hepatitis among blood donors assessed with HCV third generation tests and polymerase chain reaction. J Hepatol. 1993; 19:167–70.
crossref
23. Deville WL, Buntinx F, Bouter LM, Montori VM, de Vet HC, van der Windt DA, et al. Conducting systematic reviews of diagnostic studies: didactic guidelines. BMC Med Res Methodol. 2002; 2:9.
crossref
24. Smidt N, Rutjes AW, van der Windt DA, Ostelo RW, Bossuyt PM, Reitsma JB, et al. Reproducibility of the STARD checklist: an instrument to assess the quality of reporting of diagnostic accuracy studies. BMC Med Res Methodol. 2006; 6:12.
crossref
25. Peters JL, Sutton AJ, Jones DR, Abrams KR, Rushton L. Comparison of two methods to detect publication bias in meta-analysis. JAMA. 2006; 295:676–80.
crossref
26. Irwig L, Macaskill P, Glasziou P, Fahey M. Meta-analytic methods for diagnostic test accuracy. J Clin Epidemiol. 1995; 48:119–30.
crossref

Fig. 1.
Estimates from the studies of sensitivity and specificity of third generation anti-HCV EIA tests. Points indicate estimates of sensitivity (A) and specificity (B). Vertical lines are 95% confidence intervals for estimates.
kjlm-26-307f1.tif
Table 1.
진단검사법의 정확도에 대한 연구의 보고를 개선하기 위한 변형된 STARD 점검표
kjlm-26-307f2.tif
Table 2.
Studies including only the sensitivity of third generation anti-HCV EIA tests
Studies Reference Sample size Spectrum of patients Reference standard Test assay TP FN FP TN Sensitivity Specificity
Ré (1) [7] 22 HCV patients confirmed by PCR PCR Detect-HCV 3.0 20 2 − − 90.9 −
Ré (2)         Wiener anti-HCV 22 0 − − 100.0 −
Ré (3)         Saronno Equipar anti-HCV 21 1 − − 95.5 −
Ré (4)         Murex anti-HCV 4.0 22 0 − − 100.0 −
Ré (5)         Fujirebio Serodia-anti-HCV 22 0 − − 100.0 −
Berger (1) [8] 45 Positive or borderline samples by Abbott HCV EIA 2.0 PCR Abbott HCV EIA 3.0 24 0 − − 100.0 −
Berger (2)         Murex anti-HCV 24 0 − − 100.0 −
Berger (3)         Innotest anti-HCV III 24 0 − − 100.0 −
Berger (4)         Roche Cobas Core anti-HCV EIA 24 0 − − 100.0 −
Courouce (1) [9] 309 HCV patients confirmed by Chiron RIBA 3.0 SIA and PCR Chiron RIBA 3.0 SIA and PCR Abbott HCV EIA 3.0 307 2 − − 99.4 −
Courouce (2)         Innotest anti-HCV III 296 13 − − 95.8 −
Courouce (3)         Murex anti-HCV ver III 308 1 − − 99.7 −
Courouce (4)         Sanofi Monolisa new Ag 309 0 − − 100.0 −
Courouce (5)         Ortho HCV 3.0 308 1 − − 99.7 −
Courouce (6)         Sorin ETI-Ab-HCV 301 8 − − 97.4 −
Courouce (7)         UBI anti-HCV 4.0 301 8 − − 97.4 −
Table 3.
Studies including only the specificity of third generation anti-HCV EIA tests
Studies Reference Sample size Spectrum of patients Reference standard Test assay TP FN FP TN Sensitivity Specificity
Hennig (1) [10] 4383 Low risk blood donors Abbott Matrix HCV 2.0 Abbott AxSYM HCV ver 3.0 − − 7 4374 − 99.8
Hennig (2)         Abbott IMX HCV ver 3.0 − − 1 4380 − 100.0
Hennig (3)         Abbott HCV EIA 3.0 − − 9 4372 − 99.8
Jonus (1) [11] 3811 Low risk blood donors Chiron RIBA 3.0 SIA Abbott Modified Architect anti-HCV − − 3 3808 − 99.9
Jonus (2)         Abbott Current Architect anti-HCV − − 9 3802 − 99.8
Jonus (3)   1984 Hospitalized or diagnostic patients (random selection)   Abbott Current Architect anti-HCV − − 15 1918 − 99.2
Jonus (4)         Abbott Modified Architect anti-HCV − − 11 1922 − 99.4
Zachary (1) [12] 2020 Routine samples to virology laboratory Deciscan, RIBA, INNO-LIA Biorad Monolisa anti-HCV Plus on the Evolis automate − − 7 1931 − 99.6
Zachary (2)         Abbott AxSYM HCV ver 3.0 − − 16 1922 − 99.2
Lee (3) [13] 9936 Commercial blood donors Chiron RIBA 3.0 SIA and PCR Ortho HCV 3.0 − − 16 9846 − 99.8
Table 4.
Studies including both the sensitivity and specificity of third generation anti-HCV EIA tests
Studies Reference Sample size Spectrum of patients Reference standard Test assay TP FN FP TN Sensitivity Specificity
Vrielink (1) [14] 1923 403 Blood donors, 212 non-A, non-B hepatitis patients, 253 multi-transfused patients and 1055 blood donors PCR Abbott HCV EIA 3.0 398 0 3 1051 100.0 99.7
Vrielink (2)         Murex anti HCV VK 47 397 1 7 1047 99.7 99.3
Vrielink (3)         Ortho HCV 3.0 398 0 1 1053 100.0 99.9
Abdel-Hamid [15] 1134 Community-based longitudinal sample Chiron RIBA 3.0 SIA Abbott HCV EIA 3.0 97 1 2 1023 99.0 99.8
Ismail [16] 495 Suspicious hepatitis patients Chiron RIBA 3.0 SIA and chart review Ortho/ECi 175 0 6 311 100.0 98.1
Huber [17] 1090 Suspicious hepatitis patients PCR Roche Cobas Core anti-HCV EIA 107 7 30 946 93.9 96.9
Prince (1) [18] 301 Blood donor (ALT >100 IU/L) Chiron RIBA 2 or Abbott Matrix HCV 2.0 or PCR Abbott HCV EIA 3.0 54 0 2 245 100.0 99.2
Prince (2)         Ortho HCV 3.0 54 0 8 239 100.0 96.8
Kodama [19] 600 298 HCV patients and 302 normal people RIBA, PCR and chart review Ortho HCV 3.0 284 0 1 315 100.0 99.7
Stuyver [20] 68 Hemodialysed patients PCR Ortho HCV 3.0 23 0 0 45 100.0 100.0
Lavanchy (1) [21] 2545 400 HCV patients, 2000 routine samples and 134 tricky panels INNO-LIPA HCV Ab III Roche Cobas Core anti-HCV EIA 506 2 6 2028 99.6 99.7
Lavanchy (2)   2500 400 HCV patients, 2000 blood donors, 100 tricky panels Abbott Matrix HCV Roche Cobas Core anti-HCV EIA 401 0 0 2099 100.0 100.0
Lavanchy (3)   2545   INNO-LIPA HCV Ab III Ortho HCV 3.0 506 6 6 2022 98.8 99.7
Lavanchy (4)   2500     Murex anti-HCV 499 9 1 2033 98.2 100.0
Marcellin [22] 45 Blood donors (increased ALT) Ortho RIBA 3 and PCR Ortho HCV 3.0 12 0 0 33 100.0 100.0
Table 5.
Inter-observer agreement (%) and kappa statistics of modified STARD checklist
Item Inter-observer agreement (%) Kappa statistics
1 93.33 0.76
2 93.33 0.76
3–5 100 1
6 100 1
7 100 1
8 100 0.5
9 93.33 0.84
10 93.33 0
11 66.67 0.53
12 80 0.44
13 53.33 −0.30
14 100 1
15 100 1
16 100 1
17 100 0.5
18 100 0.5
19 86.67 0.59
20 93.33 0.81
21 100 0.5
22 100 1
23 100 0.5
24 100 1
25 100 1
Mean 93.62 0.69
Table 6.
Pooled sensitivity and specificity of third generation anti-HCV EIA tests grouped according to the type of reference test
Reference test Number of journals (studies) Total number of patients Homogeneity Estimates (95% CI)
PCR        
Sensitivity 3 (8) 2036 Yes 99.92%
      (P=1.0) (99.77–100.07%)
Specificity 2 (4) 1991 Yes 99.66%
      (P=1.0) (99.45–99.86%)

Abbreviation: CI, confidence interval.

Table 7.
Pooled sensitivity and specificity of third generation anti-HCV EIA tests grouped according to the region of index test
Region of increased reactivity Number of journals (studies) Total number of patients Homogeneity Estimates (95% CI)
NS3        
Sensitivity 8 (12) 9161 No 99.33%
      (P<0.05) (99.00–99.66%)
Specificity 10 (13) 25895 No 99.77%
      (P<0.05) (99.71–99.83%)
Core        
Sensitivity 3 (5) 7013 Yes 99.78%
      (P=0.79) (99.53–100.03%)
Specificity 2 (3) 6968 No 99.75%
      (P<0.05) (99.61–99.89%)

Abbreviation: CI, confidence interval.

TOOLS
Similar articles