PDF
Citation
Print
Introduction
Precision medicine targeting tumor-specific molecular abnormalities is increasingly playing an important role in the systemic treatment of patients with cancer [1]. Since the number of genes serving as therapeutic targets or those responsible for drug resistance has increased, the number of genes that must be tested to determine cancer treatment has also increased [2]. However, profiling tumor molecules in current clinical practice involves multiple evaluations, such as immunohistochemistry (IHC), polymerase chain reaction (PCR), Sanger sequencing, and fluorescence in situ hybridization (FISH), each targeting a single gene or mutation type, increasing the cost and turnaround time of the analyses. In addition, because orthogonal tests have an inherent limitation in that the test gene range is narrow, the information obtained can be limited. For example, for KRAS PCR testing for colorectal cancer, only codons 12 and 13 are usually tested, whereas the American Society of Clinical Oncology (ASCO) recommends additionally testing codons 61, 117, and 149 [3]. If mutations are detected, they may not respond to anti–epidermal growth factor receptor (EGFR) antibody treatment [4].
Recent advances in next-generation sequencing (NGS) analysis methods have accelerated the growth of precision cancer medicine with improvements in speed and cost reduction [5]. NGS has the advantage of testing multiple genes in a parallel manner, providing practical benefits to patients; thus, it can be used efficiently in clinical settings [6,7].
The K-MASTER cancer precision medicine diagnosis and treatment enterprise (K-MASTER project) was launched in June 2017 as a Korean national precision medicine oncology clinical trial platform that screened for actionable mutations in 10,000 patients with refractory solid tumors through NGS analysis and assigned the patients to matching clinical trials. K-MASTER carried out a nationwide gene screening master protocol (KM-00) for centralized genetic analysis and rapid clinical trial assignment. K-MASTER centralized the process of DNA extraction and NGS analysis of samples collected from each institution. K-MASTER strived to maintain the accuracy of the centralized molecular profiling by preparing its own standard operation procedures and managing the quantity and quality of tissue preparation in each institution, sample delivery, sample processing, and DNA extraction [8].
Nevertheless, there is a lack of studies comparing the data of NGS tests with that of the conventional standard test method currently used to determine treatment strategies based on actionable mutations. Therefore, we compared the results of the uniform NGS carried out in the K-MASTER project with those of the orthogonal methods currently used as a standard to determine its reliability for detecting clinically meaningful genetic alterations.
Materials and Methods
1. Patients and samples
This study included patients with colorectal cancer, breast cancer, non–small cell lung cancer (NSCLC), and gastric cancer, the largest cancer cohort enrolled in the K-MASTER master screening protocol (KM-00) between June 2017 and October 2020. Among them, only cases for which data from the clinical diagnostic test conducted by the orthogonal methods could be obtained were selected. All specimens from the subjects included in this study were prepared from formalin-fixed and paraffin-embedded tumor tissues. If possible, the most recent archived tissue of the primary tumor or metastatic site was used. The genomic DNA extraction method was described in detail in a previous study from the K-MASTER project [8]. The NGS results from K-MASTER projects were compared with those of the orthogonal method, comparing KRAS, NRAS, and BRAF mutations in colorectal cancer; EGFR, anaplastic lymphoma kinase (ALK) fusion, and reactive oxygen species 1 (ROS1) fusion in NSCLC, as well as ERBB2 positivity in breast and gastric cancers.
2. Targeted sequencing and bioinformatics
Targeted sequencing was performed using the two following NGS platforms: SNUH FIRST Cancer Panel and K-MASTER Cancer Panel. The SNUH FIRST panel v3.1 was performed with exons of 183 cancer-related genes, and the K-MASTER cancer panel v1.1 included the whole exomes of 409 cancer-related genes, the intronic regions of 23 genes, and three fusion genes. These panels used a hybrid capture method with DNA only. Detailed methods for targeted sequencing and variant annotation were the same as those described in a previous study [8]. Targeted sequencing produced approximately 100 Mb per sample, using a sampling average of 95% for each target. The sequence of all samples that achieved an average depth was more than 650× in both the FIRST and K-MASTER panels. The samples received by K-MASTER accounted for 80.8% of surgical tissues and 19.2% of biopsies. The quality control pass rate in DNA extraction was 89.1% and sequencing success rate was 96.8%.
The pathogenic single nucleotide variant was generally defined as positive when the allele frequency was at least 5%, but for the actionable variant we studied in this study, 1% or more was defined as the cutoff. Gene amplification was defined as the copy number of at least four copies.
3. Orthogonal methods
In this study, the orthogonal assay was not directly performed, and the results that had already been obtained by each laboratory were used instead. The KRAS, NRAS, and BRAF values of patients with colorectal cancer were determined by PCR. Some orthogonal methods were performed by pyrosequencing, including codons 12, 13, and 61 in the case of KRAS, and some were examined by additionally including codons 59, 117, and 146 by real-time PCR. However, some test methods were not recorded separately. For the detection of EGFR mutations, pyrosequencing or real-time PCR was used as an orthogonal method. Peptide nucleic acid (PNA)–mediated real-time PCR-based methods were performed using the PNAClamp EGFR Mutation Detection Kit (Panagene, Daejeon, Korea) or PANAMutyper EGFR Kit (Panagene) according to the manufacturer’s instructions. To confirm ALK fusion in NSCLC, IHC was performed using anti-ALK antibodies [9]. Alternatively, FISH was performed using a detachable ALK probe, and ALK rearrangements were scored as positive when more than 15% of the tumor cells exhibited split or isolated 3′ signals. To detect the ROS1 fusion gene, real-time PCR-based messenger RNA analysis was performed using the ROS1 Gene Fusion Detection Kit, licensed as a companion diagnostic. To identify ERBB2 amplification, IHC and dual probe in situ hybridization (ISH) tests were performed. ISH tests included FISH and silver ISH. The IHC test results were reported as 0, 1+, 2+, and 3+. ISH test results were reported as the mean receptor tyrosine-kinase erbB-2 (HER2) copy number and HER2/CEP17 ratio.
4. Droplet digital PCR
Genotypes with G12D, G12S, G13C, and G13D were analyzed by droplet digital PCR (ddPCR) in discordant cases of KRAS. The following PrimePCR ddPCR assays (BioRad Laboratories, Inc., Hercules, CA) were analyzed: KRAS p.G12D (dHsaCP2500596) and KRAS WT for p.G12D (dHsaCP2500597); KRAS p.G12S (dHsaCP2500588) and KRAS WT for p.G12S (dHsaCP2500589); KRAS p.G13C (dHsaCP2500594) and KRAS WT for p.G13C (dHsaCP2500595); and KRAS p.G13D (dHsaCP2500598) and KRAS WT for p.G13D (dHsaCP2500599). For each ddPCR reaction, 20 ng of DNA was used. Each reaction included a positive control (KRAS p.G12D) using the HD780 Reference Standard Set (Horizon, Cambridge, UK). All experiments were performed according to the manufacturer’s instructions, as previously described [10]. Samples that contained at least two droplets in the 6-fluorescein amidite–positive area were considered positive.
5. Statistical analysis
The data of the NGS and orthogonal platforms to detect KRAS, NRAS, BRAF, and EGFR mutations, ALK and ROS1 fusions, and ERBB2 amplification were analyzed using the chi-square test with IBM SPSS Statistics for Windows (ver. 25.0, IBM Corp., Armonk, NY). The sensitivity, specificity, positive predictive value, concordance rate, and kappa (κ) coefficient were analyzed with a 95% confidence interval (95% CI).
Results
1. Colorectal cancer cohort
Of the 1,314 patients with colorectal cancer enrolled in the KM-00 screening system, KRAS data for 224 patients, NRAS data for 197 patients, and BRAF data for 146 patients obtained using the orthogonal methods were already available. Among them, KRAS mutations were detected in 115 of 224 (51.3%), NRAS mutations in 10 of 197 (5.1%), and BRAF mutations in 7 of 146 (4.8%) patients according to the NGS report. Compared with orthogonal analysis data, sensitivity and specificity were 87.4% (90/103) and 79.3% (96/121) for KRAS; 88.9% (8/9) and 98.9% (186/188) for NRAS; and 77.8% (7/9) and 100.0% (137/137) for BRAF (Table 1). The concordance rate for KRAS was lower (83.0%) than that for NRAS (98.5%) or BRAF (98.6%). The κ index for KRAS was also lower (0.66; 95% CI, 0.56 to 0.76) than that for NRAS (0.83; 95% CI, 0.65 to 1.00) or BRAF (0.87; 95% CI, 0.69 to 1.00).
When orthogonal tests and NGS analyses for KRAS were performed with the same tissue block, the concordance rate was 86.1%, and 79.2% when different tissue blocks were used. There was no difference in the concordance rate according to the orthogonal test methods (pyrosequencing vs. real-time polymerase chain reaction, 82.8% vs. 82.5%). Further detailed analysis was performed on 38 discordant cases of KRAS. Among 13 cases where the KRAS mutation was detected using orthogonal methods but not NGS, six were reported only as a codon with a mutation without an accurate amino acid change report, and four G12V, one G12D, one G12S, and one G13D were reported in the orthogonal analysis results. Of the 25 cases where the KRAS mutation was detected in NGS but not in the orthogonal analysis, five showed uncommon codon sites that were not well covered by orthogonal PCR (three A146T, one codon 38, and one codon 184), one G12V, six G12D, three G13C, nine G13D, and one G13R.
ddPCR was additionally performed on 20 samples, excluding cases that were reported only as codons or for which a primer for performing ddPCR was not available (Table 2). In 19 of these 20 cases, each tested mutation was detected by ddPCR. In general, the ddPCR results were consistent with those obtained using NGS. The variant allele frequency reported in the NGS report showed a positive correlation with the fractional abundance of ddPCR. When the fractional abundance threshold was set to 1%, only four of the 20 cases did not agree with the NGS and ddPCR results.
2. NSCLC cohort
Of the 376 patients with NSCLC enrolled in the KM-00 screening system, EGFR data for 109 patients, ALK data for 95 patients, and ROS1 data for 42 patients obtained using single gene assays were already available. EGFR mutations were detected in 29 of 109 (26.6%) patients in the NGS report. Sensitivity and specificity were 86.2% (25/29) and 97.5% (78/80), respectively, for EGFR. The concordance rate was 94.5% for EGFR and the κ index was 0.86 (95% CI, 0.74 to 0.97). In two cases analyzed by NGS but not by orthogonal methods, exon 20 insertion and a rare mutation, G239A, were reported. In four cases analyzed by orthogonal methods but not by NGS, one showed ALK rearrangement known to be mutually exclusive for EGFR in both the NGS and orthogonal methods, and one showed de novo T790M without other activating EGFR mutations. The other two reported mutations were on exon 21, one of which was in a case of squamous cell carcinoma.
ALK rearrangement was observed in four of 95 patients (4.2%), and ROS1 rearrangement was observed in 1 of 42 patients (2.4%). Sensitivity and specificity were both 100.0% (4/4 and 91/91, respectively) for ALK and 33.3% (1/3) and 100.0% (38/38) for ROS1, respectively. The concordance rates were 100.0% for ALK and 95.2% for ROS1. Cohen’s κ coefficient was 1.00 for ALK and 0.48 (0–1.00) for ROS1 (Table 3). Clinical diagnostic data for ROS1 were only available for 42 patients, and although not shown in Table 3, ROS1 rearrangement was additionally reported in the NGS report in three of the 97 patients for whom ROS1-related clinical data were not available.
3. Breast cancer cohort
Of the 801 patients with breast cancer enrolled in the KM-00 screening system, ERBB2-related IHC or ISH data were already available for 261 patients. Among them, the ERBB2 copy number gain was reported in 45 of 261 patients (17.2%) using NGS. Table 4 shows the data of NGS for detecting ERBB2 amplification compared with that of clinical diagnostic methods. Compared with IHC, the sensitivity and specificity were 65.1% and 98.0%, respectively, if only IHC 3+ was recognized as positive. The concordance rate was 90.0% and the κ index was 0.70 (95% CI, 0.59 to 0.80). The ISH test results were available only for 89 patients because the ISH test was performed only for cases with a score of 2+ in the IHC results in clinical settings. When compared with the clinical ERBB2 status determined by combining the results of the IHC/ISH tests, the sensitivity, concordance rate, and κ index were all lower than those for IHC alone.
Thirty-eight cases showing discordant results with regard to the ERBB2 status between clinical methods (IHC/ISH) and NGS were analyzed in detail. Of these, 22 cases showed a score of 3+ in IHC, but no copy number gain in NGS. In four of these cases, IHC and NGS tests were performed on different tissues collected at different times. In two of these cases, ductal carcinoma in situ occupied a large proportion of the tumors. Of the 38 cases, the remaining 16 showed a score of 2+ in IHC and ERBB2 amplification in ISH, but copy number gain was not found in the NGS analysis. Detailed ISH test results for these cases are summarized in Table 5. Only four out of 16 cases corresponded to group 1A, with a HER2/CEP17 ratio of two or more and HER2 signals/cells of six or more. In eight cases, the HER2/CEP17 ratio was two or more, but the HER2 signals/cells were four or more and less than six, corresponding to group 1B. The HER2/CEP17 ratio was two or more, but the HER2 signals/cells were less than four (group 2) in three cases. In the remaining case, the HER2/CEP17 ratio was less than two and the HER2 signals/cells were six or more corresponding to group 3. As suggested in a previous study, only if the HER2/CEP17 ratio was two or higher and the HER2 signals/cells was six or higher (group 1A) was this considered a true positive. Only four of these 16 cases did not agree with the results of the clinical methods and NGS for ERBB2 amplification [11].
4. Gastric cancer cohort
Of the 547 patients with gastric cancer enrolled in the KM-00 screening system, IHC or ISH data for ERBB2 were already available for 64 patients. Among these, ERBB2 copy number gain was reported in six of 64 (9.4%) patients via NGS. Compared with IHC, the sensitivity and specificity were 50.0% and 94.7%, respectively, if only an IHC score of 3+ was recognized as positive. The concordance rate was 40.0% and the κ index was 0.50 (95% CI, 0.12 to 0.88). ISH test results were available for only nine patients because the ISH test was performed only for patients with a score of 2+ in the IHC results in clinical settings. When compared with the clinical ERBB2 status determined by combining the results of the IHC/ISH tests, the sensitivity, concordance rate, and κ index were all higher than those of IHC alone, unlike the case for the results of the breast cancer cohort (Table 4).
Discussion
This comparative study of NGS and gene results obtained by the orthogonal methods revealed that the degree of concordance was different for each gene, but the κ index (0.8) showed a very good performance with a concordance of 93.6% for the entire single nucleotide variation (SNV) gene mutation. These results are comparable to those of previous studies comparing NGS results with those of PCR [12–14]. In a study comparing NGS and PCR for assessing KRAS mutations in patients with colorectal cancer, a concordance rate of 90% (90/100) and a κ index of 0.794 were shown [14]. In the comparison of the results of NSCLC cases performed in China, when considering EGFR, KRAS, BRAF, NRAS, PIK3CA, HER-2, and TP53, sensitivity and specificity values of 95.2% and 77.1%, respectively, were found [12]. The concordance rate for EGFR mutation between NGS and PCR was 83.9% in this study and 92.9% in another study conducted in Korea [15].
Since genetic mutations with high incidence are inevitably low in agreement, KRAS mutation had the lowest concordance rate among the colorectal cancer cohorts. However, some of discordant cases for KRAS as positive in NGS but negative in PCR were rare codon mutations (codon 146, etc.). This is in line with the advantage associated with NGS, i.e., it is specialized in detecting a wider range of genes compared to non-NGS technologies [16]. The results of the ddPCR validation for the discordant cases of common KRAS codons were generally consistent with the NGS results, but not the PCR results. Of course, because the previous PCR analysis was not directly controlled in this study, the results might be inconsistent, and ddPCR was performed with the same sample as the NGS test, and very careful interpretation of the ddPCR results is necessary. Nevertheless, 16 of the 20 ddPCR results were consistent with the NGS results, suggesting that NGS provides more comprehensive results than the conventional methods.
For the detection of fusions, the concordance rate was substantially different depending on the gene, whereas the concordance rate of SNV was generally consistently high. Compared to the 100% concordance between NGS and FISH for the ALK fusion gene, only one of the four positive results for ROS1 fusions detected by the orthogonal methods was detected by NGS by this study. There are also known challenges in detecting fusion genes. Breakpoints for rearrangements usually occur in non-coding DNA sequences (introns) and highly repetitive regions, making it difficult to capture and map the reference genome [17]. For this reason, RNA sequencing is advantageous for detecting fusion genes [18]. However, if DNA sequencing is used, which was the case in this study, the hybridization capture-based NGS method with probes designed to capture exons/introns that span the entire gene, including the intron region or most often those involved in the fusion of interest, can improve detection efficiency [19]. It is possible that the difference between ALK and ROS1 concordance depends on the number of breakpoints that the probe contains in the DNA-based NGS test used. Previous studies regarding the detection of the ROS1 fusion gene also reported a relatively low detection rate in DNA-based NGS and recommended IHC and FISH as two complementary standard tests [20,21]. To detect gene fusions, except for ALK fusions, it is necessary to actively consider RNA-based NGS.
In this study, copy number variants (CNV) had more false negatives than SNV; thus, the sensitivity of NGS decreased slightly to 53.7% in breast cancer and 62.5% in gastric cancer for ERBB2 amplification. The detection of ERBB2 amplification also differed between breast and gastric cancer patterns. This is because the characteristics of each cancer are reflected. In breast cancer, IHC 3+ is often found, but in gastric cancer, 2+ is more often accompanied by gene amplification. The 2018 College of American Pathologists/ASCO HER2 testing guidelines recommend additional testing using HER2 IHC for unusual HER2 ISH patterns (groups 2–4) [22]. In most patients, the ISH test results in a group 1 pattern with amplification (HER2/CEP17 ≥ 2.0, HER2 signal/cell ≥ 4.0) or a lack of amplification (HER2/CEP17 < 2.0, HER2 signal/cell < 4.0). In contrast, groups 2–4 showed unusual patterns, but there remained a lack of evidence of a response to anti-HER2 treatment. According to a study by Yang et al. [11], for these unusual ISH groups, few cases showed a copy number gain in NGS among ISH groups 2–4, and even this was limited to the cases of IHC 3+. In particular, when group 1 was divided into group 1A with HER2 signals/cell ≥ 6.0 and group 1B with HER2 signals/cell ≥ 4.0 and < 6.0, there was a significant difference between 1A and 1B with regard to the ERBB2 copy number levels in NGS. All cases in group 1A showed a major copy number gain by NGS, while only one case in group 1B showed major copy number gain. In this study, since the IHC/ISH tests were performed only for clinical necessity, there were only a few cases of IHC 3+ observed in the ISH test results. Therefore, in the cases showing IHC 3+ and no detected CNVs in NGS, ISH results could not be compared. However, in discordant cases with IHC 2+/ISH + but negative NGS results, only four of 16 cases revealed ISH group 1A, which was consistent with the results of the above study.
The detection of CNVs in NGS could be underestimated or overestimated as individual sequence reads occurring in the CNV often do not show sequence changes. The relative change in DNA content would be reflected in the number of reads within the CNV region after normalization to the average reading depth in the same sample if the sequencing coverage was sufficiently deep [23,24]. A previous study emphasized that ensuring adequate tumor content and average reading depth is essential to optimize CNV detection using NGS [25]. Although this study recommended sending samples with more than 20% tumor proportions checked for each institution, unfortunately, central confirmation was not possible before NGS analysis, which could be the reason for the low concordance. Other studies cited HER2 heterogeneity in addition to low tumor content for discrepancies observed with regard to the results for HER2 in the case of each test method [26,27]. There have been many reports that intertumoral and intratumoral HER2 heterogeneity is common in breast and gastric cancers and is associated with poor prognosis [28,29]. Considering the inherent limitations of NGS, tumor heterogeneity, and the original differences between tests, it is recommended to use NGS in a complementary role with traditional tests for detecting ERBB2 amplification, rather than using NGS alone. Finally, since ERBB2 status is important as a predictor of anti-HER2 therapy, it is necessary to further study whether the degree of response to anti-HER2 therapy differs in actual patients, causing discrepancies among the results for each test.
When conventional single gene analysis and NGS tests do not match, clinicians are often confused as to which results should be accepted as the final result. Of course, since both tests have their own drawbacks, they are complementary tests, and a third verification test is required to check for inconsistencies in results. There are slight differences in which test results are more reliable depending on the variant type. Although it was the ddPCR verification of the KRAS discrepancy case, which has many limitations, as a result of this result, the NGS test result and the ddPCR concordant rate were high in the case of SNV, and in the case of CNV, the orthogonal test was still more reliable.
This study is a relatively large-scale study of a wide range of patients and genes, comparing the results of NGS performed via a uniform process and via various orthogonal gene assays for the detection of targetable genes that can be used in clinical settings. Although this study reflects actual clinical practices, the following limitations exist.
The tumor purity of the tissue slide subjected to NGS could not be confirmed at the center. This factor significantly affects the detection of structural variants, especially CNV, as well as the allele frequency of SNVs. Second, we compared orthogonal assay data without directly performing the method. Cancer cells undergo clonal evolution over time, as well as tumor heterogeneity [30]. Therefore, despite the fact that the results are inevitably different for different tissues at different time points, there is a large blind spot in that the tissues used to perform the NGS and orthogonal method assays in this study were collected at different times or at different sites. Third, even if the tests were performed using the same tissues, there were differences between the results of non-NGS orthogonal assays performed on relatively fresh tissues and NGS tests performed over time depending on the storage condition and period of the tissue. This study is the first national large-scale project of NGS for patients with cancer in Korea, and because it was registered after waiting for a long time without receiving the benefits of NGS, old tissues were often registered.
In conclusion, upon comparing the results of the K-MASTER NGS panel and conventional gene assays, the degree of agreement was different for each genetic alteration, but generally showed a high agreement rate. In particular, the total SNVs and ALK fusion genes showed very high concordance between the NGS and non-NGS orthogonal methods. ERBB2 CNVs had a higher false-negative rate in NGS than SNVs, but it is better to understand the difference in the target of each test and use it complementarily with the results of current standard test methods.
XML Download