INTRODUCTION
Thrombotic microangiopathies (TMAs) are disorders characterized by thrombocytopenia, microangiopathic hemolytic anemia, thrombosis, and the presence of schistocytes in the peripheral blood (PB) [1–3]. Thrombotic thrombocytopenic purpura (TTP) is a TMA subtype caused by severe deficiency of a disintegrin and metalloprotease with thrombospondin type 1 motif, member 13 (ADAMTS13) enzyme [4–9]. TTP has high mortality rates (up to 90%), but with therapeutic plasma exchange (TPE), mortality rates have decreased to <10% [3, 10–12].
ADAMTS13 activity was traditionally measured using sodium dodecyl sulfate-polyacrylamide gel electrophoresis (SDS-PAGE) and currently using quantitative ELISA [13]. Recently developed ADAMTS13 activity assays have shown good performance in the diagnosis of TTP, as well as high accuracy in identifying patients with a good response to TPE [3, 4, 14–22]. However, the long turn-around time and batch-type variation render ELISA unsuitable for rapid diagnosis [23]. Therefore, a diagnostic scoring system using clinical and laboratory parameters for TMA prediction is required. To date, three diagnostic scoring systems have been developed: the BENTLEY score (B-S) [24] and the French TMA Reference Center score (FTMA-S) [18] introduced in 2010 and the PLASMIC score (PLASMIC-S) [2] introduced in 2017.
The B-S was developed from a retrospective evaluation of 110 patients, including 11 patients with severe ADAMTS13 deficiency (<15% activity) [24]. The performance of the B-S for the discrimination of patients with severe ADAMTS13 deficiency (<10% activity) showed an area under the curve (AUC) value of 0.990 [25]. The FTMA-S was developed using a similar approach as used in B-S development [18]. The PLASMIC-S is based on seven components (platelet count, combined hemolysis variable, absence of active cancer, absence of stem cell transplant [SCT] or solid-organ transplant history, mean corpuscular volume [MCV], international normalized ratio [INR], and creatinine), and its performance for the detection of severe ADAMTS13 deficiency (<10% activity) showed an AUC value of 0.91–0.96 and good sensitivity, specificity, and negative predictive value (NPV) [2, 26].
Comparison of the three scoring systems in an independent cohort showed that the PLASMIC-S had superior performance compared with the FTMA-S, whereas the B-S frequently classified patients with severe ADAMTS13 deficiency in the low-risk category [27]. However, to our knowledge, a comparison of these three scoring systems using an identical definition of severe ADAMTS13 deficiency has not been performed. To address this issue, we retrospectively validated the performance of these three scoring systems in the prediction of TMA due to severe ADAMTS13 deficiency consistently defined as <10% activity and the response to TPE. In addition, we attempted to develop a method to improve the performance of the existing scoring systems.
MATERIALS AND METHODS
Study population and data acquisition
In total, 145 patients suspected of having TMA owing to the presence of schistocytes in the PB smear and thrombocytopenia who underwent ADAMTS13 activity tests from January 2014 to September 2022 at Ulsan University Hospital, Ulsan, Korea, were enrolled in this retrospective cohort study. Demographic features such as age and sex; clinical features such as active cancer and history of solid-organ or SCT before implementation of the ADAMTS13 activity test; and laboratory results included as parameters for the application of the three scoring systems, such as serum creatinine, indirect bilirubin, haptoglobin, D-dimer, platelet count, reticulocytes (%), MCV, prothrombin time (PT) INR, and fluorescent antinuclear antibody (FANA), at the time of ADAMTS13 activity test performance were collected from a retrospective review of electronic medical records. This study was approved by the Institutional Review Board of Ulsan University Hospital (approval number: 2022-11-022). The requirement for informed consent was waived because of the retrospective nature of the study.
ADAMTS13 activity assay
From January 2014 to October 2020, 91 patient samples were transported to the central laboratory of Bundang CHA Hospital, Korea, and the ADAMTS13 activity test was performed according to the standard SDS-PAGE protocol following a previously described method [9]. From November 2020 to September 2022, 54 patient samples were transported to the central laboratory test referral center (GC Labs, Yongin, Korea), and the ADAMTS13 activity test was performed using a Technozym ADAMTS-13 ELISA kit (Technoclone, Vienna, Austria). For all tests, severe ADAMTS-13 deficiency was defined as <10% activity, and patients with severe ADAMTS13 deficiency were regarded TMA-positive.
Application of the three scoring systems and risk stratification of patients for TMA according to ADAMTS13 activity status
The FTMA-S is based on the following three parameters with a score of 0 or 1 assigned for each parameter: creatinine (0 if >199.78 μmol/L and 1 if ≤199.78 μmol/L), platelet count (0 if >30×109/L and 1 if ≤30×109/L), and FANA result (0 if negative and 1 if positive). The sum of each score (from 0 to 3) was used for the risk stratification of patients for TMA. Patients were categorized as low risk for TMA if the total score was 0, intermediate risk if the total score was 1, and high risk if the total score was 2–3 [27].
The PLASMIC-S uses the following seven parameters with a score of 0 or 1 assigned to each parameter: platelet count (0 for ≥30×109/L and 1 for <30×109/L); combined hemolysis parameters (0 for indirect bilirubin ≤34.21 μmol/L, reticulocytes ≤2.5%, or haptoglobin ≥100 mg/L and 1 for indirect bilirubin >34.21 μmol/L, reticulocytes >2.5%, or haptoglobin <100 mg/L, which is considered undetectable haptoglobin corresponding to a value below the detection limit of the haptoglobin test provided by the Roche analyzer); cancer history (0 if present and 1 if absent); history of solid-organ transplantation or SCT (0 if present and 1 if absent); MCV (0 if ≥90 fL and 1 if <90 fL); PT INR (0 if ≥1.5 and 1 if <1.5); and creatinine (0 if ≥176.80 μmol/L and 1 if <176.80 μmol/L). The sum of each score (from 0 to 7) was used for the risk stratification of patients for TMA. Patients were categorized as low risk for TMA if the total score was <5, intermediate risk if the total score was 5, and high risk if the total score was >5 [27].
The B-S is calculated according to five parameters (creatinine, platelets, D-dimer, reticulocytes [%], and indirect bilirubin) as follows: (1) add−11.5 points if creatinine >176.80 μmol/L; (2) add−30 points if platelets >35×109/L; (3) add−10 points if D-dimer >21.90 nmol/L; (4) add+21 points if reticulocytes >3%; and (5) add+20.5 points if indirect bilirubin >25.66 μmol/L. Patients were categorized as low risk for TMA if the sum of the five parameter scores was <20, intermediate risk if the sum was 20–30, and high risk if the sum was >30 [27].
Development of a modified PLASMIC-S
A recent study reported a difference in the performance of the PLASMIC-S according to age [28]; therefore, we analyzed the effect of age on the predictive performance of the PLASMIC-S for confirmation. To improve the performance of the PLASMIC-S, we included lactate dehydrogenase (LDH), a well-known indicator of hemolysis [29], as an additional parameter in the modified PLASMIC-S. To compensate for inter-instrument variation, LDH was divided by the upper limit of normal (ULN; corresponding to 230 IU/L in our study). Therefore, the modified PLASMIC-S was composed of eight parameters, including the LDH/ULN ratio and the seven parameters of the existing PLASMIC-S described above. The best cut-off value of the LDH/ULN ratio to add 1 point and threshold for determining high risk in the modified PLASMIC-S were determined using ROC analysis. The performances of the LDH/ULN ratio alone and the modified PLASMIC-S for the prediction of patients with TMA due to severe ADAMTS13 deficiency when high risk is assessed were compared.
Effect of PLASMIC-S status on the response to TPE and mortality in TMA-positive patients
All TMA-positive patients in our hospital were initially treated with daily TPE with a scale of 1 plasma volume, using fresh frozen plasma as an exchange material without platelet transfusion. We assessed the response to TPE in patients with a TMA-positive status with respect to PLASMIC-S prediction. As prognostic variables, initial treatment response (ITR) achievement, defined as recovery of platelet count ≥150×109/L for more than two days after TPE [30]; number of TPE sessions required for ITR achievement; and death rates were compared between high-risk and low-to-intermediate-risk patients.
Statistical analysis
The Mann–Whitney U-test and chi-square/Fisher’s exact test (for small numbers <5 in each subgroup) were used to compare continuous and categorical variables, respectively. All continuous variables did not show a normal distribution according to a Kolmogorov–Smirnov test, and therefore, data are presented as median (range). The performances of the three scoring systems for the prediction of patients with severe ADAMTS13 deficiency (TMA-positive) when high risk was assessed by each scoring system were compared using ROC analysis. The AUC value, sensitivity, specificity, NPV, PPV, and accuracy of the three scoring systems were calculated and compared. The calculated AUC scores of two scoring systems were compared using the statistical method developed by Hanley and McNeil [31], using the MedCalc software (MedCalc Software, Ostend, Belgium). Two-tailed analyses were applied for all comparisons, and the statistical significance threshold was set at P<0.05. SPSS software version 13.0.1 (SPSS Corp., Armonk, NY, USA) and MedCalc version 9.2.0.2 were used for all statistical analyses.
RESULTS
Comparison of clinical and laboratory test results with respect to ADAMTS13 activity status
Among the 145 patients, 44 (30.3%) were TMA-positive and 101 (69.7%) were TMA-negative. TMA-positive patients showed significantly lower ADAMTS13 activity, platelet counts, haptoglobin, and D-dimer levels than TMA-negative patients. However, sex, age, creatinine, indirect bilirubin, reticulocytes (%), MCV, and PT INR were not significantly different between the two patient subgroups (Table 1).
Risk stratification based on the three scoring systems according to ADAMTS13 activity status
According to the FTMA-S, there were more FANA-positive patients in the TMA-positive group than in the TMA-negative group, whereas there were no significant differences in the creatinine levels and platelet counts according to TMA positivity and scoring status. When risk stratification was finalized, low-, intermediate-, and high-risk patients accounted for 6.9%, 47.5%, and 45.6% of the TMA-negative group, respectively, and for 0.0%, 27.3%, and 72.7% of the TMA-positive group, respectively, representing a statistically significant difference in distributions according to TMA positivity. When categorized into two risk-stratified subgroups (low and high risk), a significantly higher proportion of patients was classified as high risk in the TMA-positive group than in the TMA-negative group.
According to the PLASMIC-S, the TMA-positive group had significantly higher proportions of combined hemolysis parameter-positive patients and lower proportions of MCV/PT INR/creatinine-positive patients than the TMA-negative group. However, no significant correlations were found for platelet counts, active cancer, and history of solid-organ transplantation or SCT according to TMA positivity and scoring status, using the Mann–Whitney U and chi-square/Fisher’s exact tests. When risk stratification was finalized, low-, intermediate-, and high-risk cases accounted for 48.5%, 33.7%, and 17.8% of TMA-negative patients, respectively, and for 4.5%, 13.6%, and 81.8% of TMA-positive patients, respectively, representing a statistically significant difference in distributions according to TMA positivity. When the patients were categorized into two risk-stratified subgroups (low and high risk), the proportion of high-risk patients was significantly higher in the TMA-positive group than in the TMA-negative group.
According to the B-S, TMA-positive patients showed a significantly higher sum of points than TMA-negative patients. However, risk stratification status could not significantly discriminate TMA positivity, and an extremely small number of patients were assessed as high risk in both the TMA-negative and TMA-positive subgroups (Table 2 and Supplemental Data Table S1).
Comparison of the performance of the three scoring systems for the prediction of TMA positivity
The PLASMIC-S, FTMA-S, and B-S showed AUC values of 0.820, 0.636, and 0.513, respectively, for predicting TMA positivity when high risk was assessed. The PLASMIC-S showed a significantly higher AUC value than the FTMA-S and B-S (Fig. 1A). The PLASMIC-S also showed higher sensitivity, NPV, PPV, and accuracy (81.8%, 91.2%, 66.7%, and 82.1%, respectively) than the FTMA-S (72.7%, 82.1%, 41.0%, and 60.0%, respectively) and B-S (4.6%, 70.2%, 50.0%, and 69.7%, respectively) and higher specificity than the FTMA-S (82.2% vs. 54.5%). Although the B-S showed the highest specificity (98.0%), it also showed the lowest sensitivity (4.6%) and a low PPV (50.0%), which was because of the low number of high-risk patients in both the TMA-positive and TMA-negative subgroups (Table 3).
Effect of age on PLASMIC-S performance
When assessed as high risk by the PLASMIC-S, the diagnostic sensitivity for TMA decreased with increasing age; it was 100.0%, 78.3%, and 72.7% for ages 18–39 years (23 patients), 40–59 years (44 patients), and ≥60 years (78 patients), respectively.
Performance of the modified PLASMIC-S
The best cut-off value of the LDH/ULN ratio and threshold for determining high risk in the modified PLASMIC-S were >2.0 and >6, respectively. The LDH/ULN ratio and modified PLASMIC-S showed AUC values of 0.951 and 0.937 for predicting TMA positivity, respectively, which were significantly higher than that of the PLASMIC-S (Fig. 1B). An LDH/ULN ratio >2.0 showed high sensitivity, NPV, and accuracy (100.0%, 100.0%, and 89.7%, respectively) but slightly decreased specificity and PPV (85.2% and 74.6%, respectively). A modified PLASMIC-S >6 showed similar sensitivity and NPV (81.8% and 92.5%, respectively) and very high specificity, PPV, and accuracy (97.0%, 92.3%, and 92.4%, respectively), when compared with the PLASMIC-S (Table 4).
Comparison of the response to TPE and mortality in TMA-positive patients with respect to PLASMIC-S status
Among the 44 patients with TMA positivity, 36 patients with high risk assessed by the PLASMIC-S showed a significantly higher ITR achievement rate after TPE and a lower number of TPE sessions required for ITR achievement than the remaining eight patients with low-to-intermediate risk. The mortality rate was marginally lower in patients assessed to be at high risk by the PLASMIC-S than in those with low-to-intermediate risk (Table 5).
DISCUSSION
We found that TMA-positive patients showed significantly lower platelet counts than TMA-negative patients, and the proportion of patients with high PLASMIC-S scores, corresponding to low creatinine levels, was significantly higher in the TMA-positive group than in the TMA-negative group. These results are partly concordant with previous observations that patients with severe ADAMTS13 deficiency tend to show severe thrombocytopenia and relatively preserved renal function, which may support the selection of platelet count and creatinine level as common parameters in the three TMA scoring systems [27]. A previous study reported that the sensitivity, specificity, NPV, and PPV of the FTMA-S for the detection of severe ADAMTS13 deficiency (<5% activity) were 46.9%, 98.1%, 38.6%, and 98.7%, respectively, when all three criteria of creatinine level ≤200 µmol/L, platelet count ≤30×109/L, and positive antinuclear antibody were satisfied [18]. Our study showed that the sensitivity, specificity, NPV, and PPV of the FTMA-S for the detection of severe ADAMTS13 deficiency (<10% activity) were 72.7%, 54.5%, 82.1%, and 41.0%, respectively. The higher sensitivity and NPV and lower specificity and PPV found in our study can be attributed to the different definitions of severe ADAMTS13 deficiency and less strict criteria for the assessment of high risk.
The PLASMIC-S performed better than the FTMA-S; it showed the highest AUC value among the three scoring systems for the prediction of TMA-positive patients. These results are concordant with a previous suggestion that the PLASMIC-S has higher specificity than the FTMA-S in predicting severe ADAMTS13 deficiency, helping to avoid unnecessary TPE and to prevent wastage of blood products [27]. Compared to that in a previous study that used the same definition of severe ADAMTS13 deficiency and reported good performance of the PLASMIC-S (sensitivity, specificity, NPV, and PPV of 90.0%, 92.0%, 98.0%, and 72.0%, respectively) [26], we observed slightly lower performance, and we assume that this difference is due to variations in the features of the patient groups.
Only a small number of patients were evaluated as having a high risk of TMA according to the B-S, and the risk level did not reflect the actual TMA positivity status. In addition, 95.5% of TMA-positive patients were classified in the low-to-intermediate-risk group according to the B-S. This resulted in low sensitivity, NPV, and PPV, and false overestimation of the specificity of the B-S. These results are concordant with a previous suggestion that the B-S frequently assigns patients with severe ADAMTS13 deficiency to the low-risk group; therefore, the B-S is unsuitable as a clinical scoring system for the prediction of patients with TMA [27].
The decrease in sensitivity due to the increase in platelet count with age in TMA patients has been reported to limit the application of the PLASMIC-S [29]. In line herewith, we also observed a decrease in sensitivity (100.0% in younger age groups vs. 72.7% in older age groups) and an increase in platelet count (median: 30.5×109/L in younger age groups vs. 57.5×109/L in older age groups) with increasing age. Attempts to improve the performance of the PLASMIC-S, e.g., by including the LDH/ULN ratio [29] and proteinuria [32], resulted in improved specificity, PPV, and accuracy of the PLASMIC-S. Our results showed that the modified PLASMIC-S developed by adding the LDH/ULN ratio, which had a good performance on its own, can increase the relatively low specificity, PPV, and accuracy of the existing PLASMIC-S, in line with the previous study [29]. Because only a small number of patients underwent urinalysis, we could not evaluate the effect of proteinuria on the performance of the PLASMIC-S in our study.
Patients assessed to be at high risk by the PLASMIC-S had a better prognosis after TPE than those assessed to be at low-to-intermediate risk. This result may support a previous suggestion that the PLASMIC-S predicts mortality by identifying idiopathic TTP patients with good prognosis when TPE is applied as the initial treatment [27]. Therefore, we speculate that the PLASMIC-S may enable accurate prediction of patients who are expected to respond well to TPE. However, since the number of patients assessed to be at low-to-intermediate risk according to the PLASMIC-S was small, the statistical power is limited and our results should be interpreted with caution.
Our study had some limitations because of its retrospective design, the absence of a validation cohort, and the application of different ADAMTS13 activity test methods over time, although a good correlation (γ=0.79–0.85) between the two test methods has been reported [33]. A prospective study with a large number of patients using the same ADAMTS13 ELISA is needed to verify our study results.
In conclusion, our study showed that the performance of the PLASMIC-S is superior to that of the other two existing scoring systems (FTMA-S and B-S), and the relatively low specificity, PPV, and accuracy of the PLASMIC-S can be improved by using the modified PLASMIC-S, which includes the LDH/ULN ratio. High TMA risk determined by the PLASMIC-S may predict a good prognosis and treatment response to TPE. The PLASMIC-S therefore is the preferred scoring system for the detection of patients with TMA and for prognosis before confirmation of ADAMTS13 activity test results.