Adequate processing of tissue samples is essential in a reliable NGS cancer panel test. Required specimen handling procedures are nearly the same as those required for traditional single-gene tests. Briefly, the quality and the amount of neutral buffered formalin relative to the size of the specimen should be monitored. The time interval from specimen acquisition to fixation should be minimized, and optimal fixation duration should be monitored [9]. The optimal fixation duration depends on the dimension of each sample because formalin penetrates tissue at a rate of approximately 1 mm per hour. The rule of thumb for recommended fixation duration for surgically resected specimen is 24 hours [10].

Correct pathologic diagnosis and assessment of the fraction of neoplastic cell nuclei in tumor area by pathologists are crucial for the interpretation of NGS results. Once the diagnosis and tumor purity are established, it is strongly recommended that the pathologists circle an area from which nucleic acid is extracted so that neoplastic cells are enriched to the appropriate level of tumor purity. The minimum requirement of tumor purity can be established in the initial validation of test performances (see “Validation” section). The estimated tumor purity of a selected area should be documented. When tumor purity is assessed, the pathologists should be aware of the fact that the relative number of tumor cell nuclei reflects tumor purity rather than occupied tumor area. Thus, an area with heavy inflammatory cell infiltration should be avoided (Fig. 1A, B). Also, to ensure DNA quality, pathologists should try to avoid necrotic areas (Fig. 1C) and areas with extensive extracellular mucin (Fig. 1D).

The initial step after sequencing includes converting the base intensities in a sequencer to digital-level nucleotide sequences, called FASTQ. Although the types of signals differ among various sequencing platforms, most have their internal software for translating base calling into the compressed and de-multiplexed FASTQ files. The nucleotides in FASTQ have corresponding base quality scores that are in a form of logarithmic scales indicating the probabilistic confidence level of the bases. Appropriate quality control should be done to confirm the general integrity of the sequencing data, such as the total number of bases, sequence contents (including GC contents), per base sequence quality, etc. Read trimming is recommended if the base quality or composition does not meet the quality control threshold that each laboratory has empirically set up or that the sequencer manufacturer has recommended.

This step is to find where the short read sequences are located. In general, single or pair-end reads are first aligned (mapped) to a human genome reference. It is essential to include the version of the human reference sequence in the clinical report. Since poorly mapped sequences may lead to compromise the reliability of called variants, especially in solid tumors with low tumor content, sequencing reads should be filtered based on mapping quality score so that only confidently mapped reads are processed further. After the initial mapping, read duplicates should be removed, because unwanted clonal amplification of reads with sequence artifacts may lead to false-positive variant calls. Alignment is often challenging when the sequence reads come from genomic regions having large insertions or deletions, repetitive regions, pseudogenes, or homologous genes because there can be several other similar-looking genomic regions across the reference genome. Realignment using known references that have the suspected insertions or deletions may be necessary in such cases.

Somatic variants can be identified by subtracting variants found in non-neoplastic cells from those found in cancer. If a laboratory chooses not to sequence the corresponding normal samples, it should be noted that some germline variations might exist in the result. Laboratories should consider the implementation of modular analysis pipelines, in which different algorithms or settings are used to call different types of variants: single nucleotide variations (SNVs), insertion/deletion (Indels), copy-number variations (CNVs), and translocations.

The quality of variant calls is strongly related to the quality of alignment. The key challenge of variant calling is to distinguish real variants from sequencing errors. In general, the more times the variant is sequenced, the more reliable the variant call is. The minimum depth of coverage depends on the required sensitivity of the intended assay, the sequencing platform and the types of mutations to be detected. Although sensitivity is increased in proportion to sequencing depth, false-positive calls may also be increased especially in cases with low tumor content. There are various reasons for false-positive variant calls; they often result from PCR errors, sequencing errors, mis-mapped reads on repetitive sequences or homologous regions and so forth. Because each variant calling algorithm uses different strategies to filter out false calls, different algorithms sometimes generate discrepant results. Thus, laboratories should find optimal parameter settings during assay development and validation to minimize algorithm-dependent result variability.

Reliable identification of Indels is particularly challenging because sequence reads containing this type of variants are not often accurately mapped. Thus, the sensitivity and specificity for this type of variants is often reduced. With regard to this, laboratories should identify clinically important Indels and validate assay performances to establish reasonable sensitivity and positive predictive value for the identification of those Indels. Due to the high rate of false-positives, manual review by using visualization tools and comparison with the same regions in other samples on the same run is recommended for all Indel calls.

The reliable identification of CNV is quite difficult in NGS cancer panel sequencing because of the uneven target coverage related to hybridization capturing steps, the absence of matched normal data, or the lack of coverage uniformity [21]. Although algorithms for detecting CNV in targeted NGS tests are improving [22], the inherent limitations in cancer panel sequencing of clinical FFPE samples require robust validation of test performance. This type of validation can be done by testing characterized cancer cell lines or clinical samples with known CNV profiles (see “Validation” section) [23], although there is no agreement upon the minimal number of samples for appropriate validation.

Translocation can be identified based on the DNA level, using discordant or split sequencing reads. However, the inherent limitations of short-reads in terms of alignment can result in many false positive calls. Therefore, the test performance should be appropriately validated using reference materials with known translocations across targeted genomic regions that have been confirmed by the current gold standard. Furthermore, it is highly recommended that all translocation calls be manually reviewed by using genome visualization tools such as the Integrative Genomics Viewer. Finally, it should be noted that a translocation can be missed if the breakpoint of fusion (may be somewhere in the introns not covered by the panel) is not included in targeted genomic regions, even though protein-coding regions of the translocation partners are included in the panel.

Variant annotation determines if a sequence variant is real and provides predicted resulting amino acid changes. To identify true somatic variants, false variants should be properly filtered. Important sources of false variants include cytosine deamination, amplification errors, and sequencing errors. Cytosine deaminations are introduced ex vivo; these variants are not copied to the opposite strand, meaning that the artifacts are only present on one strand. To facilitate the detection of cytosine deamination artifacts, laboratories may use techniques such as molecular inversion probe and HaloPlex and Duplex sequencing, to enrich and sequence both the sense and antisense DNA strands [24]. Amplification errors can be introduced due to DNA polymerase errors during amplification steps of library enrichment. These errors might be minimized by the application of unique barcodes to individual DNA molecules during library enrichment. If the same variant is detected in multiple unique molecules, the variant might be real because it is unlikely that individual molecules acquire the same polymerase error during amplification. Sequencing errors are highly dependent on the sequencing platform and sequencing chemistry. The ability to call SNVs and Indels is known to be similarly accurate for data generated on the PGM and Illumina platform, provided that there is sufficient coverage [24].

Depending on the types of variants (e.g., missense, nonsense, etc.) and the types of genes (e.g., hotspots in oncogenes vs randomly distributed mutations in tumor suppressor genes), interpretation of detected variants can be simple or quite complicated. Basically, the ACMG strongly recommends that interpretation is performed by trained staffs such as clinical molecular geneticists or molecular pathologists [4]. A multi-disciplinary sequencing data analysis team with various scientific backgrounds including clinical oncology, genomics, bioinformatics, and pathology, is recommended for accurate interpretation.

For variants that are not hotspot mutations, germline variants should be excluded first. Since most cancer panel tests do not analyze matched non-neoplastic tissue, laboratories should prepare mechanisms to filter out potential germline variants based on the genetic polymorphism data on the population to which the tested patient belongs. In most instances, laboratories use public databases on germline polymorphism such as the 1000 Genomes Project, ExAC, or dbSNP (Table 1), but ideally, these data should be derived from the same ethnic group as the tested patient.

Since the clinical and biological significance of cancer-related genomic variants are increasingly characterized, many variants detected in most cancer panel tests have related information in public databases (see “Reporting” section for details). However, previously uncharacterized variants may also be detected considering the characteristics of NGS tests. Potential biological significance, or pathogenicity, could be inferred from the archived genotype-phenotype correlation data such as ClinVar, Human Gene Mutation Database, and Leiden Open Variation Database (Table 1). In addition, the in silico prediction of functional impacts is available in dbNSFP or Ensembl Variant Effector (Table 1). Since no single database is perfect, it is essential to refer to multiple resources for appropriate interpretation. Finally, knowledge about variants will be continuously improved with the accumulation of each lab’s experiences and feedbacks from clinicians.

Reporting of NGS cancer panel test results should follow the general professional organizations’ recommendations and guidelines [25]. There are two major essential parts of a report: proper patient identification and detected actionable variants. The patient identification part is the same as that in the current standard single gene-based tests, but the detected variant part is more complex and sophisticated in clinical NGS cancer panel tests because typical NGS cancer panel tests detect multiple variants at the same time. The clinical NGS report should include the essential information listed in Table 2, and the most pertinent information, such as actionable variants and a critical summary of those variants, should be placed in a clearly visible section on the first page. Detected sequence variants should be annotated in concordance with the Human Genome Variation Society mutation nomenclature and the version of the human reference sequence to which sequence reads were aligned should also be included in the report. Variants, which are not clinically actionable but are potentially useful for future practice, might follow the actionable variants. Then, detailed information about the detected variants (see the “Interpretation” section) and essential technical information, such as genes or genomic regions included in the panel and key quality control metrics, may be listed. Inclusion of granular details of technical information is not recommended, but a description of how clinicians can obtain the details may be included [26].

Since typical cancer panel results include several cancer-related genomic variants with different levels of clinical or biological evidences, variants should be classified and reported according to the level of evidences. Several leading institutions use their own variant classification system and each institution may discuss with clinicians about which variant classification system should be used. However, a recently published related guideline is a good example [27]: (1) Tier 1, strong clinical significance such as Food and Drug Administration (FDA)–approved therapies, professional guidelines, prognostic and diagnostic markers based on well-powered studies with expert consensus; (2) Tier 2, potential clinical significance such as FDA-approved therapies for different tumor types, investigational therapies, prognostic and diagnostic markers based on small published studies with some consensus and preclinical trials or case reports without consensus; (3) Tier 3, unknown clinical significance; and (4) Tier 4, benign or likely benign. Tier 1 to 3 variants should be included in the clinical NGS report in a decreasing order of clinical significance and it is not recommended to include Tier 4 variants in clinical NGS report.

Interpretation of detected variants in terms of their clinical impact and pathogenicity is a daunting task. As mentioned in the “Interpretation of computational output” section, many information sources such as public databases, published guidelines, and computational prediction algorithms should be integrated for proper interpretation. In addition to the previously mentioned public databases on germline polymorphism, in silico prediction of the functional impacts, genotype-phenotype relationship, and several cancer-specific mutation databases are available online (Table 1). These resources are very useful in the interpretation of variants in the context of cancer, but it is important to make sure that the database is properly curated, referenced and updated in a regular basis. Regarding this, FDA guidance suggested that appropriate databases should implement decision matrices with published details of each variant’s interpretation and have documented standard operating procedures (SOPs) for the curation and update of this information [28].

When assigning variants to each tier, it is essential that they should be interpreted in the context of both variant details (e.g., BRAF V600E) and patient tumor type (e.g., malignant melanoma vs colorectal cancer). This is because different variants in a single gene may have different clinical impacts and the clinical significance of a particular variant may vary depending on the tumor type. For tier 1 variants, a clear statement on their clinical relevance and any appropriate action should be included. Also, it might be helpful to clarify the absence of other variants that are important in the patient tumor type. For tier 2 variants, advising the ordering physician about available clinical trials may be useful. In addition, tier information is subject to change as new evidences and therapeutic options are continuously emerging.

In cases where only limited interpretation can be made and full quality control standards could not be met, it is essential that pathologists make a professional judgment on whether the result should be reported. Also, any limitation of the analysis should be clarified in the report. For example, negative results in samples with low neoplastic cell content at near or below the established sample rejection criteria should be accompanied by a note on potential false negatives. Furthermore, clinical NGS laboratories should have clearly defined protocols about when additional confirmatory tests should be advised, as well as performance validation data in cases where those confirmatory tests are not necessary.

The detailed documentation of SOPs is critical for quality assurance of a complex, multi-step wet bench process. All SOPs of each step of the wet bench process must be documented so that each step can be traced. This includes documentation of all methods, reagents, instruments and controls (if applicable). Most of the documentations should be similar to those of current standard single gene testing, but those specific to NGS testing include detailed information regarding captured regions, such as genomic coordinates of captured probes and lists of genes and target enrichment protocols. Clinical laboratories that process different types of samples, such as FFPE samples or blood, should establish SOPs for each validated sample type. Metrics for quality control to assess run status must also be documented. Examples include mean target coverage, percentage of reads that map to target regions, and the fraction of bases meeting specified quality and coverage thresholds. Laboratories must define and document acceptance or rejection criteria for each step of the wet bench process, such as DNA extraction, library preparation, and sequencing.

Before testing patient samples, clinical NGS laboratories must establish the analytical validity of the intended tests. If the intended test is approved by the Ministry of Food and Drug Safety (MFDS), laboratories can verify the performance specifications established by the manufacturer. If the intended test is not approved by the MFDS, i.e., a laboratory-developed test, laboratories must establish performance characteristics such as accuracy, precision, sensitivity, specificity, and reportable range.

Because NGS cancer panel tests involve complex, multistep processes, each step needs to be empirically optimized to determine optimal assay conditions. Once those optimizations are done, an analytic validation should be performed for a whole test in a “beginning to end” fashion. Test performance should be separately validated for different types of samples, such as FFPE or blood.

For “beginning to end” validation, a number of samples should be analyzed to assess the test performance. There is no general agreement on how many samples are required for this type of validation. With regard to this issue, the CAP concluded that adding a minimum number of samples for validation is premature given the ongoing evolution of NGS technologies and the diversity of diagnostic applications [3]. Literature review revealed that the sample number for validation varied with a range of 20–80 samples [29-32]. The minimum number of samples needed for an appropriate validation might be affected by the size of the test, i.e., a larger gene panel will require more extensive validation, and by the range of samples to be tested, i.e., FFPE samples with variable tumor content. Here, we suggest the general principles of validation in terms of several important analytic performance parameters.

Considering the inherent characteristics of NGS cancer panel tests, it is important to evaluate as many different genomic regions as possible because sequence context can influence sequencing results. In addition, laboratories should determine analytic performances for all variant types relevant to the intended test (e.g., SNV, Indel, CNV, and translocations). Since NGS-based tests interrogate multiple variants at the same time, the validation of test performances involves two parts: method-specific (detection of as many variants as possible in a single sample) and analyte-specific (detection of a certain genomic variant in multiple samples). Laboratories can establish the test performances by combining those two approaches.

Accuracy validation is a “method-specific” way of validation. For this type of validation, laboratories might use well-characterized cancer cell line samples to interrogate whether the intended test accurately detects all known variants within the genomic regions covered by the panel. In this case, reference cell line samples should be periodically monitored for identity and passage number to prevent a significant genetic drift. In addition, laboratories might use well-characterized HapMap samples (NA12878 and NA19240), variants of which are readily available on the web [33,34]. Details of available reference materials are summarized in Table 3. Evaluation of these reference samples might be quite useful to validate the accuracy across a wide range of genomic regions [23]. During this type of validation, an adequate depth of coverage threshold [35] necessary to accurately call all expected variants can be empirically established by plotting false-positives and false-negatives as a function of coverage.

Analytic sensitivity can be evaluated by comparing NGS test results with the current gold standard test results for known genetic variants in reference materials. To this end, laboratories may use historical controls such as accumulated clinical samples with well-characterized genomic variants by gold standard methods, provided that the gold standard tests were done in an appropriately accredited environment.

Analytic specificity is theoretically calculated by determining the fraction of test negatives (wild type sequence calls by the intended test) per true-negatives (samples that are known to have wild type sequences by the gold standard method). However, this concept often does not work well for NGS-based tests because too many potential variants are included in typical cancer panels. For most laboratories, it might be reasonable to leverage specificity by determining the average number of false-positive calls for the regions tested in a number of well-characterized clinical samples.

The limit of detection can be determined by cell line “mixing” experiments, in which a cell line with known genetic variants is serially diluted with another cell line without such variants to have different mixture ratios, which are then tested to determine whether the intended test accurately detects all specified variants under a certain coverage threshold. Alternatively, laboratories may determine the limit of detection by using the HapMap samples (NA12878 and NA19240) mentioned earlier by mixing them in various ratios and interrogating variants that differ between the two different HapMap samples.

Precision (repeatability for intra-run and reproducibility for inter-run variability) can be determined by repeating the same samples in the same run (repeatability) or by repeating the same sample in a different run (on a different day by a different operator). Ideally, testing at least three samples is recommended to adequately establish precision adequately [3].

Any changes to a clinical NGS test, such as changes in instruments, specimen types, reagents, or software, require that performance specifications be re-established or be shown to be unchanged. For example, inclusion of new genes to an existing gene panel requires revalidation to make sure that new sequence variations are reliably detected without compromising the quality of the original assay. The extent of re-validation depends on the predicted influence on the test performance related to the change. For example, if only the bioinformatics pipeline is updated, it may not be necessary to re-validate all steps before data analysis.

Clinical NGS laboratories should establish and follow a quality management plan. This plan should be integrated within the institution’s overall quality assurance program. Components of the NGS quality management program are not much different from the traditional single gene-based tests. Once laboratories establish an initial validation of test performances, laboratories must perform internal quality controls daily and external quality controls periodically. Each component of the quality assurance program specific to NGS cancer panel tests is discussed in detail below.

As NGS cancer panel tests with appropriate quality controls are reported to meet a clinical grade performance [23,44-47], routine confirmatory testing is not recommended. However, laboratories should have a policy that clearly documents both any indication for confirmatory testing as well as any performance validation data upon which they decided that such confirmatory testing is not necessary. In addition, the CAP is flexible regarding the methods used for any needed confirmatory testing.

Keeping comprehensive laboratory records is essential in monitoring complex, multi-step NGS cancer panel tests. In this regards, such records should be maintained in such a way that all detailed information about test procedures including reagents, sequencing runs, wet lab, and bioinformatics procedures and responsible technicians is traceable. While all details need not be included in the clinical report, laboratories should maintain a database from which detailed information regarding the analysis of individual specimen can be obtained.

Laboratories must be prepared for upgrades to make sure that they are not using obsolete methods. A policy for the upgrade of instruments, sequencing chemistries, and reagents or kits, as well as subsequent post-upgrade validation of test performances should be in place. The policy may include specified intervals for upgrade and required validation processes, depending on the type or extent of upgrade.

Laboratories should document all bioinformatics processes, including all data files, variant caller’s parameters, and versions of the bioinformatics algorithms. Sources and versions of all bioinformatics algorithms should be documented and updated properly. Quality control information on bioinformatics analysis, such as the cut-off of read depth, base quality score, and mapping quality, should also be documented.

General validation principles were already discussed in the Validation proportion of the “Wet Bench Analytic Process” section. Briefly, for bioinformatics pipelines, laboratories should iteratively find parameters for optimal performance of computational algorithms before applying it to the lab process. Once the pipeline is initially validated, variations between sequencing runs should be monitored daily. Principles of the quality management were already discussed in the “Wet Bench Analytic Process” section. In short, laboratories should monitor any deviation from established performance characteristics in terms of quality metrics and analysis results. For any deviation, laboratories should document the investigational measures and corrective actions made to resolve the deviation. Essential quality metrics for bioinformatics performance verification include depth of coverage, uniformity of coverage and base call quality scores. In addition, GC bias, proportion of reads that map to nontargeted regions, and percentage of duplicated reads could also be used to monitor performances of sequencing reaction and subsequent bioinformatics analyses (Table 5).

The bioinformatics pipeline should be revalidated upon any changes in operating systems, software, or overall pipelines, which may otherwise affect its analytic performance. Since a huge amount of data files are generated from the bioinformatics pipeline, it is impractical to store all sorts of files considering the significant cost. Instead, the CAP NGS workgroup recommended that some important file formats, such as FASTQ, BAM, and VCF, should be stored for quality controls or investigational use [3]. There is no general agreement on the required storage period, but it is important for laboratories to set their own storage policies in accordance with local or national requirements (if any) and inform clients of those policies.