PDF
Citation
Print
External quality assessment (EQA) is a widely accepted method for evaluating clinical laboratories and enhancing their testing quality [1]. EQA helps laboratories recognize and resolve their deficiencies in routine processes while instilling employee confidence [2]. All laboratories should therefore be encouraged to participate in EQA schemes, and such participation should be mandatory wherever possible [3]. Effective participation in EQA schemes in Europe is a mandatory requirement for country-specific accreditation bodies to have access to International Standards Organization (ISO) 15189 accreditation [4, 5]. In the United States, laboratories that conduct moderate or high-complexity tests are subject to reported inspections on a biennial basis and should participate in an EQA scheme authorized by the Center for Medicare & Medicaid Services under the Clinical Laboratory Improvement Amendment Law, which applies to all laboratories testing human specimens [6]. In Korea, laboratories with a satisfactory EQA can receive a quality incentive for testing since the notification of the Ministry of Health and Welfare took effect in 2017 [7]. However, EQA participation is not yet mandatory for laboratories in Korea, except for referral laboratories, and even basic data such as the adequacy of EQA schemes are not available. Since many EQA schemes vary broadly in terms of content, it is crucial that the EQA organizers share their experiences to continuously improve the EQA scheme. We provide an overview of the EQA schemes and performance evaluation criteria of the Korean Association of External Quality Assessment Service (KEQAS) and suggest directions for joining global harmonization movements.
The KEQAS has been the leading authorized EQA institute for the standardization and quality management of clinical laboratories in Korea since 1976. Although the number of KEQAS programs is relatively small compared with other major EQAs, all the most requested routine tests, except special tests performed only at some university hospitals, are covered by the existing programs. The KEQAS obtained ISO 17043 (EQA provider) accreditation in August 2015. Major changes to the EQA schemes were implemented in 2016; the assessment is now fully computerized to accelerate feedback, and the methods of analysis and reporting across schemes are unified [8] (Table 1). Since these changes, the number of programs has increased significantly from 46 in 2016 and 65 in 2019 to 70 in 2020. These programs cover all disciplines of laboratory medicine, including three programs of accuracy-based proficiency tests, two of point-of-care tests, one of liquid biopsy, and three of next-generation sequencing, with a total of 852 test items covered and 1,844 institutions participating in EQA as of February 2020. Approximately 50% of hospitals (including small-to-medium sized hospitals, general hospitals, and tertiary care hospitals) that submit health insurance claims for laboratory tests in Korea participate in the KEQAS EQA [9]. Currently, specimens for 50 programs are prepared inhouse, whereas specimens for the remaining programs are purchased from third-party manufacturers. With respect to the transport time after specimen shipments (e.g., the sixth shipment of 2019), 90% of the participating laboratories received the specimens within 32 hours and 99.9% within 48 hours. EQA results may be influenced by the deterioration of specimens during transportation and storage before testing [10]. Many specimens should be transported refrigerated or frozen; therefore, it is advantageous to deliver the specimens as soon as possible.
Accuracy-based EQA, which refers to commutable materials with target values, has substantially contributed to improving the accuracy of clinical laboratory tests [10]. The KEQAS has provided accuracy-based EQA for HbA1c tests since 2009, and for five chemistry tests (cholesterol, HDL cholesterol, LDL cholesterol, triglyceride, and creatinine) since 2011 [11]. The number of participants in accuracy-based EQA for HbA1c and creatinine in 2020 was 597 and 1,758, respectively. In 2011, Miller and colleagues [10] suggested six categories of EQA based on specimen characteristics, including commutability, value assignment method, and replication in the EQA survey. For example, programs in category 1 use commutable specimens with target values established by a reference system, and programs in category 2 are the same as those of category 1, except that specimens are not replicated within the survey cycle. Programs in categories 3 and 4 use commutable specimens, but the target values are not assigned by a reference system. Programs in categories 5 and 6 use non-commutable specimens [10]. Accuracy-based EQA in the KEQAS belongs to category 2, whereas the other KEQAS programs belong to category 6. The KEQAS should not only continue to increase its accuracy-based programs, but should also attempt category 1 EQA, which allows for evaluations of imprecision in laboratories by conducting repeated tests.
The consensus value of a peer group is the basis of a laboratory’s evaluation by the KEQAS EQA scheme. A peer group usually consists of laboratories that use the same analyzer from the same manufacturer, as similar matrix-related bias for a given specimen can be assumed. The use of manufacturer-based peer groups is the only acceptable method for comparing the test results of multiple analytes in immunoassay, hematology, and molecular test schemes, which lack standardization and/or harmonization across participating laboratories that use similar principles but slightly different methodologies [12]. The peer groups are further divided into instrumentor reagent-based subgroups. However, for general chemistry, the peer group is based on those using the same methods, not on the same manufacturer, because many laboratories use an open system with regards to the manufacturers of instruments, calibrators, and reagents. The peer groups are further divided into reagent manufacturer-based subgroups. The KEQAS evaluates the participants’ results based on the standard deviation index (SDI) among peer groups for quantitative tests, which is calculated as the difference between the individual laboratory test results and the mean result of the peer group divided by the peer group SD. Therefore, the SDI reflects bias as a multiple of the SD. The SDI is evaluated when the peer group size (i.e., the number of participants in each category) is eight or larger after removing outliers. In such cases, the subgroups are also evaluated. SDI > 3 is considered unacceptable.
Currently, there are large differences in the analytical performance specifications (APS) used in different EQA schemes [13]. Maximum tolerance limits can be statistically determined (e.g., using ± 3 SDIs or Z-scores) or established as fixed percentages or amounts (e.g., ± 15% of the target value or ± 10 mg/dL) [14]. As the SDI is a standardized value, it can be compared among all analytes [15]. However, the limitation of the statistical method is that when applying the SDI as a tolerance limit, the acceptable range for peer groups with larger SDs is larger than that for peer groups with smaller SDs. Quantitative responses of the US College of American Pathologists (CAP) EQA scheme are evaluated based on a fixed range, mean percentage, SD, or variable range according to the test items. Switzerland’s Suisse de Contrôle de Qualité uses government regulations and a combination of limits established by scientific societies and Z-scores to determine the acceptable range. The Netherlands’ Dutch Foundation for Quality Assessment in Medical Laboratorie (SKML) and the UK Welsh EQA provider (WEQAS) use a combination of biological variation and state-of-the-art methods [13]. Although the KEQAS has been using the SDI as a tolerance limit for evaluation, APS should be considered as an alternative based on the clinical requirement for each test.
Peer groups of qualitative tests are formed in the same manner, that is, according to the same instrument manufacturer and the same reagent manufacturer with respect to the characteristics of the tests. Flow diagrams of performance evaluation for qualitative and semi-quantitative tests are shown in Fig. 1A and B, respectively.
The performance evaluation for qualitative and semi-quantitative tests has not yet been standardized [13]. For qualitative tests, 80% consensus of referees or participants is the standard used for evaluation in the US CAP EQA scheme; for example, in urinalysis dipstick tests, 80% participant consensus can be determined by grouping the mode with the next one or the two most frequent responses. In the EQA scheme of the UK WEQAS, the spiked values are used to determine the target value; if these values are not available, interpretation is based on the majority percentage of responses from participants. In the EQA scheme of SKML, performance is scored using a point system based on expert findings or consensus results. However, detailed information on the evaluation criteria of these EQA schemes are not available. The KEQAS’s new suggestions for performance criteria for qualitative and semi-quantitative tests based on experience will be useful for achieving global EQA harmonization.
Two reports (the participant evaluation report and participants’ summary) are electronically generated simultaneously within five working days after each participant submit its results for each round of the scheme (Fig. 2). The mean turnaround time from result submission to report release was 33 days (range 6–104 days) in 2019 because of the review by the program manager. One of the drawbacks of EQAs is that laboratories cannot obtain feedback in a timely manner [6]. Therefore, KEQAS should consider ways to shorten the time for the review. For example, the evaluation criteria should be well established, there should be measures in place to cope with exceptions, and the assessment should be fully automated.
Approximately 60%–70% of the laboratory tests errors are due to the pre-analytical process [16]. Therefore, identifying appropriate quality metrics is crucial in determining the quality of laboratory services [17]. According to the model of quality indicators developed by the Working Group of the International Federation of Clinical Chemistry and Laboratory Medicine [17], proficiency testing and EQA schemes have allowed clinical laboratories to measure, monitor, and improve their analytical performance over time [18-20]. It may be helpful to introduce extraanalytical quality indicators in the KEQAS EQA scheme to monitor and improve the overall quality of more laboratories.
In conclusion, the KEQAS has been providing the EQA scheme for 45 years to improve the quality of clinical laboratories in Korea. Our summary of the EQA scheme, performance evaluation criteria of the KEQAS, and suggestions for improvement would help achieve global harmonization of EQA.
XML Download