Total cholesterol concentration is used to diagnose dyslipidemia and determine the risk factors for cardiovascular diseases. The recent National Cholesterol Education Program (NCEP) guidelines [1] set specific medical decision points for serum total cholesterol (e.g., desirable concentrations: <200 mg/dL [5.2 mmol/L]; high and undesirable concentrations: >240 mg/dL [6.2 mmol/L]). The medical decision points for total cholesterol concentration were based on national population studies conducted by the United States Centers for Disease Control and Prevention (CDC) and were standardized to the decision points determined using the Abell-Kendall reference method [2]. To use these decision points, clinical total cholesterol concentration measurement procedures must be standardized and harmonized within clinically meaningful limits. Using results that are neither standardized nor harmonized may lead to significantly erroneous clinical decisions. Manufacturers have made various efforts to prepare products for measuring total cholesterol concentration accurately for routine clinical use. However, accurate total cholesterol concentration measurement remains challenging with respect to traceability to the reference material, altered matrix characteristics of calibrators and controls, and other factors [2].

The degree of standardization and harmonization of total cholesterol concentration measurement procedures can be evaluated by the external quality assessment (EQA). EQA is a widely accepted tool for monitoring and improving method performance in clinical laboratories and plays a central role in achieving trueness verification standardization and harmonization [3]. The Korean Association of External Quality Assurance Service (KEQAS) has been the nation’s top authorized EQA institute for the standardization and quality management of laboratory tests in Korea since its foundation in 1965 [4]. The KEQAS was certified as an EQA provider according to ISO/IEC 17043 in 2015 [5].

Only a few studies have evaluated the accuracy of total cholesterol concentration measurement worldwide using EQA, and no such studies have been conducted in Korea [6, 7]. We report the total cholesterol concentration measurement standardization status of products from various manufacturers using KEQAS data analysis for the first time in Korea; our findings can indirectly indicate reagent performance.

This retrospective study involved 1,670 laboratories that participated in the KEQAS assessment of 32 products from different manufacturers for measuring total cholesterol concentration in 2019. Total cholesterol concentrations in three quality control (QC) materials (samples A, B, and C) were measured in the first round of KEQAS in February 2019. The samples included a human serum powder (Bio-Rad Laboratories, Hercules, CA, USA) and were shipped to the participating laboratories in a box with ice packs under refrigeration. The total cholesterol concentration in each sample was measured using routine methods without modification of routine practice. The laboratories reported their results, methods, and instruments used.

The target values of the three QC materials (A, B, and C) were measured at the National Medical Reference Laboratory (NMRL) of the Korea Disease Control and Prevention Agency (KDCA), a member laboratory of the Cholesterol Reference Method Laboratory Network that participates in an EQA scheme for reference laboratories of the Joint Committee on Traceability in Laboratory Medicine (JCTML). Total cholesterol concentration was measured by isotope-dilution gas chromatography-mass spectrometry (ID/GC/MS), which was certified by the JCTML, and then converted to the predicted Abell-Kendall method value using a previously reported equation [8-11]. The target values of the three QC materials were determined as the means of three replicates.

The mean and SD of the total cholesterol concentration measurements reported by different manufacturers are summarized in Table 1. The KEQAS acceptable range (≤3 SD), target value measured at the NMRL of the KDCA, and the performance criteria are shown in Fig. 1. The NCEP-based performance criteria of samples A, B, and C by the NMRL were 254.65±7.64, 108.30±3.25, and 256.29±7.69 mg/dL (6.59±0.20, 2.80±0.08, and 6.63±0.20 mmol/L), respectively. The total mean values of samples A, B, and C in the KEQAS data from 1,670 laboratories were 254.5, 103.7, and 255.3 mg/dL (6.58, 2.68, and 6.60 mmol/L), respectively. Values in 42.7%, 68.4%, and 38.0% of the laboratories using samples A, B, and C, respectively, were outside the performance criteria range. In contrast to those for samples A and C, the total mean value of sample B was outside the performance criteria range. The acceptable ranges for all manufacturers were wider than the range for the target value performance criteria. Additionally, in the case of a few manufacturers, the peer group means were outside the target value performance criteria range. Three manufacturers using sample A, nine using sample B, and four using sample C had a peer group mean outside the performance criteria range. At lower concentrations (sample B), participating laboratories showed poor accuracy compared that at high concentrations (sample A and C). In the case of peer groups with high SD, even if the peer group mean falls within the performance criteria, many laboratories were found to be out of the performance criteria because the acceptable range was wide as 3 SD.

Although all manufacturers have tried to maintain traceability to standard material, the actual inaccuracy of products from various manufacturers was revealed by comparing the true values of QC materials used in the KEQAS. This approach of using the peer group mean as the reference has the potential to create errors when evaluating EQA results. Although the KEQAS data, which are based on peer group mean and 3 SD, suggest that <0.27% of laboratories were outside the allowed range, this study, based on true values and performance criteria, showed that a significant number of laboratories did not meet the performance criteria (42.7% using sample A, 68.4% using sample B, and 38.0% using sample C). The acceptable ranges for all manufacturers in the KEQAS were outside the range of performance criteria. A high SD group has a high probability of deviating from the performance criteria, indicating that a high SD manufacturer is more vulnerable to QC than other manufacturers.

The KEQAS results show that, despite efforts to standardize total cholesterol concentration measurement worldwide, differences in test results still exist across manufacturers and can lead to serious errors in the practical application of various clinical guidelines, including those of the NCEP. There are a variety of factors underlying errors in test results performed in clinical laboratories, and analytical errors depend largely on the manufacturers. The reproducibility of the equipment and the traceability of the calibrator affect the test results. In approaches using peer group mean, analytical performance depends on the precision of the peer group using the same manufacturer’s product, since the peer group mean is regarded as the target value. Therefore, as long as manufacturers only deal with reproducibility issues, the EQA results of user laboratories are not of concern. Statistically, only a small number of laboratories outside the 3 SD (<0.27%) will obtain unacceptable results; thus, biases originating from a specific manufacturer cannot be detected by interpreting the EQA results using the peer group mean. For globally standardized tests such as total cholesterol, accuracy-based EQA can detect bias and imprecision simultaneously because of the target value of the QC materials.

There are many options for analytical performance criteria based on clinical outcomes, including the clinician’s expert opinion, biological variability, and accrediting agency. We selected the performance criteria based on published professional recommendations [1, 12].

Although QC through the KEQAS has improved in many clinical laboratories in Korea, additional efforts are required to keep pace with the recent standardization of lipid testing worldwide. Monitoring EQA result trends and providing feedback to individual manufacturers are also necessary. One of the factors underlying the differences between manufacturers may be calibrator traceability. Some calibrators have been identified as untraceable; even those marked as traceable they show different values when measured using a standard procedure.

Overall industry performance can be elucidated using data from these accreditation and regulatory agencies (such as the KEQAS). However, the manufacturers in this study were not ranked, because the test results were not determined solely by the manufacturer’s products. Pre-analytical elements, such as technicians, transportation, vial-filling imprecision for the QC materials, varying storage stability, shipping, and in-use stability of QC materials owing to handling at user laboratories could also affect the test results. This study has a limitation that a commutability test to exclude matrix effects was not performed [13].

In conclusion, we demonstrated the biases between each peer group means and target values in total cholesterol concentration measurements. EQA data represent a useful source for comparable analysis of in vitro diagnostics. Although significant efforts have been made to accurately measure total cholesterol concentrations to date, further actions are needed by manufacturers to standardize total cholesterol concentration measurement and determine calibrator traceability accurately.