PDF
Citation
Print
INTRODUCTION
Alpha-fetoprotein (AFP) is a glycoprotein mainly produced in the yolk sac and liver cells during the embryonic stage. With continued research interest in understanding the functions of serum AFP and continuous improvement in measurement methods, the serum AFP concentration has been found to be significantly elevated in patients with various cancers. Related findings have identified AFP as a sensitive serum marker for diagnosing primary hepatocellular carcinoma (PHC). AFP is widely used for early screening, diagnosis, monitoring disease development, guiding treatment decisions, and prognosis predictions for patients with PHC [1-5].
In human serum samples, AFP is measured using various automated systems in clinical laboratories, most commonly by performing chemiluminescent immunoassays or electrochemiluminescence immunoassays. The accurate measurement of AFP is crucial for decision-making regarding the diagnosis of relevant diseases and patient management. Metrological traceability refers to the value of laboratory results obtained with a certified reference material (RM) and reference-measurement procedure, achieved with a series of calibrations [6]. The AFP RM 72/225, designated by the WHO as a high-order RM, is used to assign values to manufacturers’ calibrators with an uninterrupted chain of traceability [7, 8]. Ensuring RM commutability in the traceability process is imperative [7]. One contributing factor to the lack of agreement is the use of non-commutable RMs as calibrators in the calibration hierarchy for clinical laboratory-measurement procedures. Commutability is defined as equivalence in the mathematical relationships between the results of different measurement procedures for a control material and representative clinical samples (CSs). Commutability of control materials is one of the most important factors affecting the implementation of external quality assessment (EQA) programs. Employing commutable materials enables the transfer of laboratory performance in such programs to CSs [9]. Ideally, an EQA program involves control materials that are commutable with patient samples and have values assigned using higher-order reference methods [10]. Providers of reference and control materials should be definitively responsible for assessing the commutability of such materials before use [11].
To standardize AFP measurements, we assessed differences and commutability with the WHO 72/225 standard diluted with six different matrices, using five different AFP immunoassays, and selected the appropriate WHO 72/225 diluent matrix to improve the comparability of AFP measurements in the different assays. We also examined the optimal approach for deciding whether prepared candidate RMs (Can-RMs) for AFP exhibit suitable commutability for use with EQA or proficiency-testing samples, which would support their use as common calibrators for measurement procedures or trueness control materials provided by manufacturers to validate calibrations [12].
MATERIALS AND METHODS
Instruments and reagents
The five automated immunoassays and reagents used in this study were as follows: Abbott Architect i2000 (Abbott Diagnostics, Abbott Park, IL, USA), Beckman DxI 800 (Beckman Coulter Inc., Brea, CA, USA), Roche Cobas E601 (Roche Diagnostics GmbH, Branchburg, NJ, USA), Mindray CL-2000i (Mindray Co., Ltd., Shenzhen, China), and Autolumo A2000 (Autobio Co., Ltd., Zhengzhou, China).
The commercial diluents were phosphate-buffered saline (PBS) (Na2HPO4, KH2PO4, NaCl, and KCl; pH 7.2–7.4), glucose-free DMEM (a14430-01; Gibco, Grand Island, NY, USA), minimal essential medium (MEM; 51200-038; Gibco), and Roswell Park Memorial Institute 1640 (RPMI) medium (r7509-500 mL; Sigma-Aldrich, St. Louis, MO, USA). A Milli-Q water system was obtained from EMD Millipore (Billerica, MA, USA). In addition, a healthy human serum pool prepared from leftover patient serum samples (collected from Beijing Chaoyang Hospital) was used as a diluent for WHO 72/225 and had a mean native AFP concentration of 3.0 µg/L.
The standard material used for AFP analysis was WHO 72/225 (60,500 µg/L), which was purchased from the National Institute for Biological Standards and Control (Hertfordshire, United Kingdom).
Preparation of Can-RMs
We collected 40 leftover patient serum samples with low, medium, and high AFP concentrations from the Laboratory Department of Beijing Chaoyang Hospital, where the AFP concentrations ranged from 5.1 to 730.9 µg/L (as measured using the Roche Cobas E601 system). Each sample (at least 2.5 mL) was evenly divided into five aliquots, which were stored at −80°C until further use. This study was approved by the Ethics Committee of Beijing Chaoyang Hospital (approval number 2023/7/11-2) and was performed in compliance with the Declaration of Helsinki. The requirement for informed consent was waived due to the retrospective nature of the study design.
Based on the CLSI C37 guidelines [13], human serum pools without hemolysis, lipemia, and icterus were prepared using serum samples from the Laboratory Department of the Beijing Chaoyang Hospital as leftover samples for AFP measurements. The samples had different AFP concentrations and were directly collected into tubes and frozen at −80°C daily. The frozen serum aliquots were thawed at room temperature, pooled, and analyzed using a Beckman DxI 800 analyzer in 1 day. These pools were thoroughly mixed, filtered through 0.22-µm membranes, aliquoted into 2 mL cryogenic vials (1 mL/vial), and stored at −80°C for later use as trueness controls or for EQA.
WHO 72/225 standards
The WHO 72/225 standard material (60,500 µg/L) was diluted in six different matrices, including PBS (P), a healthy human serum pool (S), DMEM (D), MEM (M), RPMI 1640 (RP) medium, and distilled water (DW) to theoretical standard concentrations of 726.0, 363.0, 181.5, 90.8, 36.3, and 18.2 µg/L. These samples were labeled as P1–P6, S1–S6, D1–D6, M1–M6, RP1–RP6, and DW1–DW6, respectively. Each diluted WHO 72/225 sample was divided into five equal portions. Samples from the same batch were analyzed using the following five immunoassays on the same day: Roche Cobas E601, Beckman DxI 800, Abbott Architect i2000, Mindray CL-2000i, and Autolumo A2000. All samples were analyzed in triplicate.
Commutability study
According to the CLSI EP30A guidelines [14], six concentrations of the WHO 72/225 standard diluted with six different diluent components and six levels of Can-RMs were randomly allocated among the 40 individual serum samples, and all samples were measured in triplicate with the five assays in 1 day. For commutability evaluations, the measurements were transformed to a logarithmic scale, which stabilized the SDs. The transformed data were analyzed using Deming regression, and 95% prediction intervals were calculated for each pair of assays. The commutability of the AFP RMs was evaluated based on whether the logarithmic value of the RM was within the prediction interval for CSs, measured in paired assays [8, 14, 15]. Commutability assessments were also performed according to the difference in the bias analysis based on the recommendations of the IFCC Working Group on Commutability [12, 16]. The difference between the bias for an RM and the average bias for clinical samples (BCSs) was denoted as the dRM, and the standard uncertainty of the dRM, the U(dRM), was calculated based on the distribution of the mean of bias for patient samples over the entire concentration interval [12]. The maximum |dRM| value for the RM was designated as the commutability criterion (C). The U(dRM) from the commutability sheet regarding the difference in bias between the CSs and the RMs is represented for each RM with error bars. The uncertainty consisted of two components: the uncertainty of the estimate of bias for the CSs and the uncertainty of the estimate of bias for each RM. When the uncertainty interval was within BCS±C, the RM was commutable; otherwise, it was non-commutable, and when it overlapped with the C limit, the result was inconclusive [16].
Value assignments for Can-RMs
The WHO 72/225 standard was diluted in PBS (P1–P6) and Can-RMs (L1–L6) and then measured in the same analytical sequence in triplicate on 2 consecutive days using the five assays. For each assay, WHO 72/225 was used as a common standard for traceability and transmission of the measured value. The linear-regression results for the theoretical concentration of the diluted WHO 72/225 standards and the actual measured concentrations were plotted pairwise. The assigned values of the Can-RMs were calculated after standardization with the WHO 72/225 standards. The values were assigned for Can-RMs using the average results from the five clinical analyzers.
AFP EQA survey
The Beijing Center for Clinical Laboratories (BCCL) sent trueness-verification samples (Can-RMs L2 and L3) to 45 Beijing Municipal Health Commission-affiliated laboratories for AFP measurements, and each sample was analyzed thrice on 3 consecutive days by each laboratory. Nine AFP measurements were obtained for each sample in each laboratory, and the AFP measurement results obtained from the analysis were recorded or tabulated. Precision (CV), trueness (bias), and accuracy (TE) data were used to evaluate the measurement performance, and the tolerance limit derived from the desirable biological variation (https://www.westgard.com/biodatabase1.htm) was used to evaluate the acceptability of the results [17, 18]. Within-subject and between-subject biological variations were expressed as percentages [19]. The difference between the tested and target values was determined as the bias. CVs were calculated from the nine replicate results, and the TE was calculated as the bias±2 CV [20].
Statistical analysis
The commutability of all RMs was assessed based on CLSI and IFCC approaches. Regression analyses were conducted using Deming regression, and graphs were prepared using Microsoft Excel 2010 (Microsoft, Redmond, WA, USA). Bias, CV, and TE evaluation criterion were calculated from data on biological variation (https://www.westgard.com/biodatabase1.htm) to evaluate the results of the EQA program. Intralaboratory CV was calculated from the 9 repeated results, and TE was calculated as the bias+2CV. Accordingly, the optimal analytical quality specifications of the measurements were 5.9%, 3.1% and 10.9% for AFP, respectively; the desirable analytical quality specifications of the measurements were 11.8%, 6.1%, and 21.9% for AFP, respectively; the minimal analytical quality specifications of the measurements were 17.7%, 9.2%, and 32.8% for AFP, respectively.
RESULTS
Comparability of the diluted WHO 72/225 standards in different AFP immunoassays
Diluted WHO 72/225 RMs (samples P1–P6, S1–S6, D1–D6, M1–M6, RP1–RP6, and DW1–DW6) were assessed using five immunoassays. The deviations of the measured values for the diluted WHO 72/225 samples from their standard concentrations of 726.0, 363.0, 181.5, 90.8, 36.3, and 18.2 µg/L were calculated. The results of all five assays for WHO 72/225 prepared in the six different buffer matrices are shown in Fig. 1. The measured concentrations of 72/225 in each buffer differed with each assay system. When PBS, DMEM, MEM, and RPMI were used as diluent buffers, the largest positive deviations between the measured values and the standard concentrations for WHO 72/225 were observed with the Roche and Mindray assays, which overestimated the AFP concentrations. The Abbott and Beckman systems showed negative deviations and underestimated the AFP concentrations. The Autolumo system showed a trend from a positive deviation to a negative deviation. In the serum buffer matrix, only the Mindray assay showed a positive deviation, and the Autolumo system showed a relatively significant variation. In the DW matrix, a trend toward a negative deviation was observed with decreasing AFP concentrations with the Beckman, Abbott, and Autolumo assay systems (Fig. 1).
Commutability of the RMs for AFP measurements
Fig. 2 shows the regression curves for the AFP measurements in 40 individual serum samples using the five assays, according to the CLSI method. Can-RM samples L1–L6 (developed by BCCL) were commutable across all five assays in 10 pairwise comparisons. In addition, the WHO 72/225 standard was commutable after dilution to six concentrations in all buffers, except for the serum matrix (Autolumo vs. Roche) and DW matrix (Abbott vs. Roche/Mindray). However, the Can-RM L1 and L2 results were inconclusive for the Roche vs. Autolumo comparisons, and the Can-RM L6 results were inconclusive for the Roche/Abbott/Beckman vs. Mindray comparisons, based on the IFCC method (Fig. 3). With the IFCC approach, the biological variation of the AFP data generated was 17.7% (bias), which was used as the C value. The WHO 72/225 standard diluted in different RMs showed commutability among 4 of 10 assay combinations in PBS, 5 of 10 assay combinations in serum, 2 of 10 assay combinations in RPMI, 3 of 10 assay combinations in DMEM, 4 of 10 assay combinations in MEM, and 5 of 10 assay combinations in DW. The commutability results for Can-RMs L1–L6 and the WHO standard diluted at six concentrations were determined using both the CLSI and IFCC approaches (Table 1 and Figs. 2 & 3).
Final assigned values of the BCCL AFP Can-RMs
The average assigned AFP concentrations for the BCCL Can-RMs obtained using five assay platforms were calculated based on calibration curves by interpolation from linear regression curves. Each result was expressed as the standard value ± total uncertainty. The total uncertainty of the standard value reflected the uncertainties introduced by the measurement uncertainty, homogeneity, long-term stability, and the uncertainty of international RMs. The factors contributing to the uncertainty, including those introduced by assignments with multiple systems, were comprehensively calculated for the obtained results. The final assigned concentrations for the BCCL Can-RMs L1–L6 were 12.0±0.7, 27.0±3.7, 84.2±8.1, 126.0±8.9, 232.2±21.9, and 666.4±97.3 µg/L, respectively (Table 2).
Analysis of the EQA AFP trueness program
The CV, bias, and TE evaluation criteria were determined based on the desirable level of biological variability and were 6.1%, 11.8%, and 21.9%, respectively. Using these values, the pass rates were calculated separately. When testing two levels of samples, the percentages of laboratories passing the desired bias were 74% (Can-RM L2) and 87% (Can-RM L4). In addition, 96% and 98% of the laboratories met the desired CV for Can-RM L2 and L4, respectively, and the percentages of laboratories meeting the allowable TE were 85% and 96%, respectively. For all participating laboratories, the mean bias was 1.2% (range, −17.6%–19.7%) for Can-RM L2 and −1.5% (range, −18.5%–13.1%) for Can-RM L4 (Table 3).
DISCUSSION
The purpose of an EQA program is to provide a service enabling laboratories to enhance their analytical performance to satisfactory levels and improve the quality of the results. Most routine EQA programs use materials without verified commutability and use consensus means (either from a peer group or all laboratories) as target values [10]. According to ISO17043, proficiency-test items should match in terms of the matrix, measurands, and concentrations [21]. The commutability of EQA samples with CSs is important in laboratory quality evaluations [11, 22]. In addition to using commutable RMs, it is necessary to assign values (and uncertainty) to the RMs and to define and apply clinically permissible analytical-performance specifications to substantiate the validity of laboratory measurements in clinical settings [9, 23].
We evaluated measurement differences when the WHO 72/225 standard was diluted in PBS, serum, DMEM, MEM, RPMI, or DW at six concentrations (18.15–726 µg/L). We also examined how the measured concentrations of the WHO 72/225 standard diluted in the same buffers differed with different assay systems. We calculated the deviations of the measured values at six concentrations from the standard concentrations of the diluted WHO 72/225 standards. The results revealed different deviations in the measured values with all five assays for samples in all six buffers at different concentrations. Consistent trends in terms of positive and negative deviations were observed for all five assay systems, with PBS, DMEM, MEM, or RPMI serving as the dilution buffer. Differing deviations were observed in the different assays when serum or DW was used as the matrix. These findings demonstrate how the measured AFP concentration may be affected by the diluent selected for the WHO 72/225 RM [8].
Currently, no recognized reference method exists for AFP. The commutabilities of the BCCL Can-RMs and WHO 72/225 RM across six diluent matrices were analyzed using five routine assays from different manufacturers according to the CLSI EP30A guidelines, and the difference bias was evaluated using the IFCC approach. The commutability of the tested samples differed between both approaches. The BCCL Can-RMs were commutable according to the EP30A criteria but provided inconclusive results for IFCC-based bias estimations in the Autolumo/Mindray vs. Roche and Abbott/Beckman vs. Mindray system comparisons. The WHO 72/225 RM, when diluted in PBS, DMEM, MEM, and RPMI, exhibited commutability across all six diluted concentrations for the five measurement systems with the CLSI approach. However, inconclusive or non-commutable results were obtained for some immunoassay comparisons with the IFCC approach.
Linear regression with prediction intervals (according to the CLSI approach) has been commonly used to assess the commutabilities of different RMs. This approach is based on the statistical distribution of differences in the results obtained with CSs between two assays, where the precision may vary among different pairs of assays [12]. This approach does not quantify how closely the RM agrees with the average relationship for the CS at the concentration of interest [16]. The commutability results for the RMs obtained correlate weakly with clinical data. In terms of assessing differences in bias, the standard for evaluating commutability is that the deviation/difference ± uncertainty of the RM should fall within the fixed range of the allowable deviation. The same judgment criteria can be used to evaluate all measurement procedures. However, the evaluation standard should be selected carefully and avoid being too strict for methods to pass the evaluation. Differences in the uncertainties of RMs can also affect commutability evaluations [24]. The main sources, such as the number of replicates performed with CSs and Can-RMs, position effects, and sample-specific differences (which all contribute to the uncertainty), can be specified. This approach can result in uncertainty in the commutability assessment in that the outcome may become inconclusive or indicate non-commutability [25]. Many factors can lead to non-commutability, such as the nature of the analyte and its matrix and the principles of specific assays [26]; thus, further investigation is necessary.
We assessed the commutability of the international standard material WHO 72/225 with five different AFP immunoassays (Abbott Architect i2000, Beckman DxI 800, Roche Cobas E601, Mindray CL-2000i, and Autolumo A2000). The measured mean values of the five assay systems were used as the assigned values for six concentrations of Can-RMs. Two concentrations of Can-RMs were selected as EQA samples for use with the trueness-verification program in Beijing. An average of 95% of testing laboratories met the requirement for intra-laboratory CV, and 81% and 91% met the requirements for bias and TE, respectively, indicating that more laboratories met the tolerance limit for precision (CV) than the tolerance limits for trueness (bias) and accuracy (TE). Accordingly, bias may be the main limiting factor for accurate and comparable AFP results. However, owing to the limitation of having only a few specific concentrations of samples available, it was challenging to conduct assay comparisons across the entire range of assays [27].
A major limitation of this study is that only five assays were used for AFP measurements. The second limitation is that only two concentrations were used for the EQA samples, which did not cover the entire measurement range. Future validation studies with more assays and EQA samples covering more concentrations are required. The third limitation is that the individual samples for the commutability study were stored at −80°C until use. Further research is needed to confirm whether the commutability outcomes were affected by protein-structure changes caused by freezing.
In conclusion, the commutability results for the RMs obtained with both evaluation approaches differed. Owing to differences in their intended use, both evaluation approaches involve different ways of evaluating the commutability, which may explain the observed variations in evaluation results. Specific diluents for the WHO 72/225 AFP standard are needed to ensure accurate value transmission in the traceability process. The Can-RMs exhibited good commutability and, thus, are suitable for use as commutable EQA materials with the values assigned and might improve the performance of laboratory AFP measurements. The trueness-verification results provide reliable data for promoting the improvement of the corresponding quality specifications.
XML Download