PDF
Citation
Print
I. Introduction
Cancer refers to the uncontrollable multiplication and growth of cells that are either localized (carcinoma in situ [CIS]) or have spread to other parts of the body. Cervical cancer, although treatable when detected early, is the most common form of cancer in women aged 35 and younger [1]. The prevalence of cervical cancer is disproportionately high in individuals who have sexual relationships at early ages, who have sexually transmitted infections, and who use tobacco [1,2]. Therefore, cervical screening is highly recommended for women aged between 21 and 65 years once every 3 years [3,4]. However, as many people have skipped screening appointments and avoided visiting healthcare facilities due to infection-related fears amid the recent coronavirus disease 2019 pandemic [5], concerns have grown regarding the possible demand for healthcare resources once the pandemic is over [6].
There is a gradual progression leading to cervical cancer, from normal squamous (NS), to dysplasia (or precancerous cells), and finally to invasive disease. Cytological screening for cervical cell abnormalities using the Papanicolaou (Pap) smear is an effective method to detect precancerous cells. These precancerous stages (or cervical intraepithelial neoplasia [CIN]) are classified as (1) CIN I or low-grade dysplasia (LGD), (2) CIN II or high-grade dysplasia (HGD), or (3) CIN III or CIS, depending on the extent to which squamous cells fail to mature as they migrate to the surface of the epithelium [7]. Dysplastic cells have nuclei that occupy a large fraction of the intracellular compartment and display nuclear membrane irregularities. There is also an increase in the nuclear-to-cytoplasmic ratio (NCR) from 1:7 in normal squamous cells to 1:1.5 in the CIN III group [8]. Therefore, the cell features used by cyto-technicians in the diagnosis include aspects of cell appearance such as size, color (after staining), textures and shapes, NCR, and the distribution of nuclei [9]. However, not only is this approach subjective and prone to inter- and intra-observer variability, it requires experience and a trained eye, and is time-consuming.
As internet-accessible databases continue to grow, accompanied by advances in computing techniques (e.g., parallel programming and edge computing), these resources have contributed to major milestones in artificial intelligence research. At present, most of the computer-aided diagnostic (CAD) systems used for cervical cancer detection require the extraction of engineered features, such as shape-based features [10,11], morphological features [11], and statistical features [12], used by a classifier in its diagnosis. Classifiers such as decision forests, artificial neural networks, K-nearest neighbor models, and support vector machines (SVM) have been used with considerable success for binary or multiclass problems in various domains. Among these classifiers, SVM has received relatively more research attention and has been adopted in various areas [13–15]. This model has also been tested for its superiority in cervical cell recognition [16,17]. This superiority is attributed to its robustness to noise and outliers, its high susceptibility to overfitting, and its capability to handle nonlinearly separable data.
There is growing progress in the use of pretrained deep networks such as LeNet [18], Inception-ResNet [19], Visual Geometry Group network (VGGNet) [20], ShallowNet [21], and deep ConvNet [22], which can be applied to cervical cell grading through transfer learning. These models were reported to provide a higher classification accuracy than training a neural network from scratch [23,24]. At present, most research has studied only two-class problems: normal and abnormal [18,21,22]. Those studies have reported accuracies nearing 100% using a hybrid of deep learning and handcrafted features. A major effort that is worth mentioning is the fusion technique highlighted by Jia et al. [18]. Their study demonstrated that a combination of features from various layers of a convolutional neural network (CNN) and strong features (textural, morphological, and chromatic) extracted from cervical cell images enhanced prediction accuracy (up to 99%) using an SVM-based model. A similar approach was adopted in another study [25] that observed a lower average accuracy of approximately 80% using whole slide images. This method of feature fusion can be a tedious and time-consuming process, as it involves the laborious task of manual optimization and iterative feature engineering.
Since different treatment regimens should be used for lesions of different dysplasia categories [26,27], a CAD system able to diagnose multiple lesion types would enable clinicians to establish the most appropriate therapeutic strategies and prevent the risk of progression to advanced stages. For this reason, researchers [24] extended early efforts [17,19,23] and studied up to four-class problems with NS, LGD, HGD, and CIS. We sought to expand this line of research by including cervical columnar cell types as an additional condition. This type of cell has borderline risk (i.e., it is neither normal nor with evidence of dysplasia [28]), so it is placed in a separate class of problems. The main contributions of this paper include: (1) a simplification of earlier attempts using binary partitions of extracted deep features for the classification task, (2) a five-class cervical cell image classification using joint CNNs and an error-correcting SVM method, and (3) a discussion of key factors affecting the performance of classification systems and future improvement opportunities. This work was carried out using MATLAB 2020a.
II. Methods
Below are presented brief descriptions of the images used and the image processing steps carried out in this study. Sections II-2 and II-3 deal with the methodologies used in our work.
1. Data Handling and Processing
We carried out our research on a benchmark dataset containing 917 single cervical-cell images downloaded from Herlev University (http://mde-lab.aegean.gr). This is, to our best knowledge, the only public database that contains a comprehensive number of cervical cell types. This data source allowed us to assess the performance of our models relative to the task requirements and facilitated comparison with previous research that used the same dataset [16,18–22]. This dataset comprises seven labeled classes of cervical cells with clinical outcomes commonly examined in cervical cancer research: normal (i.e., superficial, intermediate, and columnar squamous) and abnormal (i.e., mild, moderate, and severe dysplasia, and CIS). These images are grouped, in our paper, into the following five classes, as discussed above: NS, columnar squamous (CS), LGD, HGD, and CIS (Figure 1). Polygonal-shaped superficial and intermediate cells represent normal cervical epithelial cells (NS) [19]. They are found in abundance in the superficial and intermediate layers of the squamous epithelium, and their relative numbers vary depending on the stage of the ovulation cycle. Cases of mild dysplasia are classified as LGD, while cases of moderate and severe dysplasia are grouped under HGD in accordance with the Bethesda system. These classes had an imbalanced data distribution, as shown in Table 1. They were randomly split using a random seed number of 1 and a data partition ratio of 0.6:0.15:0.25 for training, validation, and model testing, as shown in Figure 2. The training data size was enlarged using an augmentation strategy to prevent overfitting. This was done by rotating the original training images by 10°, −10°, 20°, −20° angles and vertically flipping the images. These color images were resized to 227 × 227 × 3 and 224 × 224 × 3 according to the acceptable input size for the different network models used in this work.
2. Convolutional Neural Network Models
The deep CNN models used in our experiment for supervised learning were AlexNet, VGG19, and ResNet50 due to the simplicity (in the case of AlexNet) and uniformity (i.e., VGGNet) of their architecture. ResNet is a stack of residual blocks that solves the vanishing gradient problems with skipping connections (dashed lines) for better training.
We first trained the pretrained CNN models with all layers frozen for the cervical cell classification task with the training dataset (including augmented images) in Table 1. The softmax loss function was used as the activation function in this end-to-end training process (as shown at the center of Figure 2). These models were tested and fine-tuned using a validation set that was not part of the training set to provide evidence of over- or under-fitting of the data during each training session. The final training was performed with the sdgm optimizer, 30 training epochs (N), a mini batch size (β) of 32, and a gradient threshold (ψ) and initial learning rate (η) of 0.01 and 0.008, respectively. This was the best hyper-parameter set determined from a fixed grid search space. In the experiment, 35 iterations were performed to test different parameter sets randomly combined from: optimizer: {adam, sgdm}, N = {10, 20, 30}, β = {20, 32, 64}, ψ = {0.01, 0.1, 1}, and η = {0.001, 0.005, 0.008, 0.01, 0.05, 0.1}. We then extracted the learned features from the first fully connected (fc) layer in each trained model for further training using the SVM classifier system to enhance the model’s classification performance. The code was run on an Intel Xeon E52680. Each model was consecutively trained three times and the average run time was recorded.
3. Joint CNN-SVM Method
The SVM method is based on the construction of single or multiple hyperplanes to solve nonlinear problems via matching attributes of observations to a class label. The SVM technique used in this research was combined with the error-correcting output codes (ECOC) model (using the fitcecoc function in MATLAB) to decompose a multiclass classification problem into several binary ones. Although some of the algorithm’s parameters can be configured by users to optimize the classification performance, the default settings were used in this study. The ECOC method uses (k(k–1))/2 learners for k classes. Each binary learner is trained on feature vectors of the training data of two class labels, as shown in Table 2, using SVM. For example, in the case of learner 1, all observations in the CIS class were grouped into a positive class (+1), observations of HGD into a negative class (−1), and those that belonged to other classes were ignored before this learner is trained on them. This is repeated for the remaining learners. During the classification of a new (unseen) observation, the outputs of the classifiers are combined to form an output code. This code was compared to the 10-bit codes in Table 2. The difference was optimized using the aggregated losses of the quadratic function, L, given in Equation (1) [29]. The class with the nearest code to the output code was assigned as the class label.
where n is the sample size, w
j
is the weight for observation j, y
j
is the corresponding class label and f(X
j
) is the classification score for observation j of the predictor data X. We used this multi-learner classifier to complement the softmax function in the original network, which was reported to be ineffective at minimizing intra-subject variation [30], and to enhance the decision regarding the diagnosis. This classifier system used the features (from fc6 in AlexNet and VGG, and from fc in ResNet) in this second-stage training of the CNNs. The results of our experiment showed a higher mean prediction accuracy (approximately 4%) using this technique than using softmax alone. The largest increases were recorded as 2%, 4%, and 9%, respectively, using the trained AlexNet, ResNet-50 and VGG19, and SVM jointly. In this paper, only results from the SVM classifier are reported in detail.
III. Results
The classification performance of CNN-SVM models used in this work was compared using the mean accuracy, specificity, and recall or sensitivity measures given in Equations (2)–(4). While accuracy describes the closeness of the prediction to the ground truth, both specificity and sensitivity quantify the efficiency of a model in differentiating the classes of cervical cells. Table 3 shows our models’ performance evaluated on the testing dataset and average simulation run time for three consecutive training sessions.
where M denotes the total class labels (M = 5). A true positive (TP
i
) is a class member correctly predicted as a specific class label, i. A false positive (FP) is a nonclass member incorrectly predicted as a class member, a false negative (FN) is a class member misclassified as a nonclass member, and a true negative (TN) is a nonclass member correctly classified as a nonclass member.
Before moving on to present the results from each model in Table 3, it is worth mentioning that we noticed cases of false negatives (testing images of other classes were being misclassified as NS) by VGG19-SVM and ResNet50-SVM in every run (i.e., a non-repeated image from the 226 images in each run), but no such cases were seen using AlexNet-SVM. Highly similar values of the evaluated performance metrics between the models can also be observed from this table. Therefore, the best-performing models (shown in bold in Table 3) were further investigated and discussed. Their confusion matrix and receiving operating characteristic (ROC) curves (i.e., performance graphs) from a one versus all analysis are plotted in Figure 3. The performance comparison based on sensitivity and specificity measures in Figure 4 showed the same adequacy of the models in detecting the NS class label, while the poorest performance was seen in the HGD categories. The area under the curve (AUC) for each diagnosis class showed values ranging from 0.85 to 1, as shown in Figure 4, indicating the overall high reliability of our diagnostic system in differentiating normal from abnormal cells and in the grading of dysplastic images. A comparison of the performance of our method with that of other recently reported approaches is shown in Table 4.
IV. Discussion
In this study, we adopted different transfer-learned CNNs combined with an SVM classifier system for the classification of cervical Pap smear images. Our findings, shown in Table 3, suggest good consistency in both intra- and inter-model comparisons of classification performance. The ResNet slightly underperformed as compared to the more complex VGGNet, suggesting that the recovery of the lost information (in ResNet) in the training phase may not significantly improve the accuracy.
Even though all the models used in the present study accurately identified normal cervical cases (with excellent specificity and sensitivity of 100%, and an AUC of 1), the VGGNet and ResNet models exhibited some cases where cervical cell images of other classes were misclassified as normal. An investigation of these images showed that they had overlapping cells, poor contrast, and cropped cells at image borders. These FN results are a matter of concern, as the patient would miss the optimum time for treatment that would maximize therapeutic outcomes. Meanwhile, the AlexNet-SVM model demonstrated good estimation performance, in terms of high sensitivity and specificity, good reproducibility, and fast training speed, as shown in Table 3. We observed a similar classification performance between AlexNet-SVM and VGG19-SVM (overall accuracy >80%; mean AUC = 0.95), with the former requiring only a fraction of the computational time in the training (by an order of magnitude) than other models. Both models demonstrated good performance in the classification of CIS and HGD (i.e., specificity and sensitivity ≥80%), as shown in Figure 4, but they had considerably inferior sensitivity performance (approximately 60%–70%) in the NC and LGD classes. Our results showed that several NC images (approximately 25%) were misclassified as HGD, as shown in Figure 3. This resulted in high FP rates (about 20%) in HGD and lower AUC values (0.86–0.89) than with other class labels. Similarly, about one-third of LGD images were misclassified as NC and HGD. This likely occurred because of gradual and minute changes in the characteristics of these cervical cell images (e.g., nuclei size, NCR) as the lesion progressed, as shown in Figure 1. Some features important for the diagnosis may not have been sufficiently and consistently captured in the employed dataset. For example, some missing cell borders and variability in the nuclei size ranges (possibly due to inconsistent cropping) were identified in most of the images used. The cell characteristics in the current dataset exhibit subtle and complex differences, which can be difficult to discern using the current architectures. Instead, irrelevant information may have passed through the networks. This was aggravated by the increased number of considered class labels, which reduced the overall classification performance, as has also been reported in previous research [20,25]. This may prompt a search for an enhanced network architecture, such as the use of extra convolutional or pooling blocks to preserve more relevant features from primary feature maps in the near future. The classification performance may also be enhanced by using a dataset with consistent imaging conditions (e.g. brightness, exposure/illumination, and viewing conditions) and fine-tuning of hyperparameters, which can make the process less laborious, while also covering a larger search space using an optimization algorithm.
Most of the recently reported studies in Table 4 have recommended introducing either adaptive feature selection or fusion into deep neural network models to enhance classification performance. However, this strategy can be experimentally exhaustive and time-consuming. Our research simplifies the earlier designs and focuses on using deep features of trained CNNs and the SVM model for classification tasks. The comparison between methods in Table 4 shows that the architecture and use of CNNs have a significant influence on classification performance. From this table, we identified the experimental group and design as the two key factors affecting prediction accuracy. The closest work to ours, which considered up to four-class problems [24], observed a higher accuracy (>90%), which can be attributed to the use of a larger and independent dataset.
In conclusion, based on our analyses, AlexNet-SVM is preferable for this task. This model can be easily and conveniently extrapolated to clinical practice to support the diagnoses made by professionals. Our system has comparable efficacy to that of state-of-the-art systems in identifying normal cells, thereby eliminating unnecessary biopsies to confirm the diagnosis. Meanwhile, more precise dysplasia grading may be achieved with the use of a meticulously designed network and by including a larger dataset with consistent imaging conditions in the training process.
XML Download