PDF
Citation
Print
INTRODUCTION
Colorectal cancer (CRC) is a global health concern, ranking as the second most frequently diagnosed malignancy worldwide, with an incidence of 1.36 million cases each year [1,2]. Moreover, the global occurrence of CRC has been steadily rising, with an annual increase of 3.2% [3]. Metastases from CRC pose a significant obstacle to curative treatment, representing a pivotal factor contributing to CRC-related mortality [4]. Amongst the organs susceptible to CRC distant metastasis, the liver is the most frequently affected [5]. Population-based studies have revealed that 25% to 30% of CRC patients experience colorectal cancer liver metastases (CRCLM) throughout the course of the disease [6,7], as a result of lower gastrointestinal portal venous drainage [8]. While only 25% of patients with CRCLM qualify for operative resection [9], advancements in the field have expanded the treatment options for CRCLM. Early detection and accurate prediction of CRCLM are paramount to improving prognosis and delivering appropriate care for CRC patients.
The application of predictive modeling techniques in healthcare has brought about a transformative shift in the analysis and interpretation of medical data [10,11]. These advanced computational approaches offer the potential to uncover intricate patterns and relationships that may remain latent within large datasets, eluding traditional statistical methods [12]. Within the realm of CRCLM, the amalgamation of predictive modeling algorithms facilitates the development of robust prognostic tools that can consider the intricate interplay of multiple variables, providing individualized predictions [13-16]. These predictive models offer heightened accuracy by incorporating clinical parameters, pathological characteristics, and molecular biomarkers, empowering clinicians to make informed decisions on treatment strategies and patient management [17].
Despite numerous individual studies exploring the application of predictive models for CRCLM, a comprehensive evaluation of existing literature is currently lacking. A systematic analysis of the evidence is therefore timely.
MATERIALS AND METHODS
Database and search strategy
This study was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA, 2020) guidelines [18]. A comprehensive literature search was conducted in May 2023 using the following terms: (“machine learning” OR “machine intelligen*” OR “machine vision*” OR “artificial intelligen*” OR “deep learning” OR “neural network” OR “supervised learning” OR “unsupervised learning” OR “reinforcement learning” OR “predictive model*” OR “predictive model*”) AND (“colon*” OR “rectal” OR “colorectal” OR “colonic” OR “rectum” OR “bowel” OR “intestine”) AND (“cancer*” OR “malignan*” OR “neoplas*” OR “tumor*” OR “tumour*”) AND (“liver meta*” OR “liver metastasis”) in the PubMed, MEDLINE, Embase, and Web of Science databases.
Eligibility criteria
Studies were included if (1) the study population comprised male or female patients aged 18 years and above; (2) the participants consisted of adult individuals with CRC and liver metastasis; (3) the studies explicitly described and employed predictive modeling techniques to forecast the occurrence of CRCLM; (4) the studies reported the performance metrics of the predictive models, including sensitivity, specificity, accuracy, or the area under the receiver operating characteristic curve (AUC); (5) the full text of the articles was available for analysis; and (6) the articles were written in the English language. Case reports, reviews, and meta-analyses were excluded. The selection criteria did not specify a preferable study design or setting.
Study selection
Two reviewers (YZ and BHA) screened the titles and abstracts of the articles identified in the search to identify potentially eligible studies. Full-text articles were obtained for all potentially eligible studies, and were independently reviewed by these two reviewers to determine eligibility. Any discrepancies were resolved with a third author (ISE) through discussion and consensus. The study selection process was documented using a PRISMA flow diagram.
Data collection and analysis
The data collection was structured as per the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies (CHARMS) [19]. The information extracted from each article included: the first author’s name, publication year, country of the study, study design, surgical approach, sample size, model details, model performance, and model evaluation. Due to significant heterogeneity observed in the study design, model development, and validation methodologies across the included studies, a meta-analysis was not performed.
Risk of bias assessment
The risk of bias (RoB) of the selected studies was assessed using the Prediction Model Risk of Bias Assessment Tool (PROBAST) [20]. This tool consists of four domains: patient selection, predictors, outcomes, and analysis. In addition, another evaluation was conducted to assess the applicability of the included studies across three domains: participants, predictors, and outcomes. Two independent reviewers (YZ and BHA) assessed the RoB for each domain, and assigned a rating of high, low, or unclear RoB. Any discrepancies were resolved through discussion with a third author (ISE). The RoB assessment was documented using a graphical summary.
RESULTS
Overview of predictive modeling techniques
Table 1 presents a summary of CRCLM predictive modeling techniques included in the review. In broad terms, predictive modeling encompasses statistical learning, machine learning, and deep learning approaches (Fig. 1) [21-23]. Statistical learning primarily involves developing and applying statistical methods and algorithms for predictive purposes. Notable examples of statistical learning methods include logistic regression (LR), least absolute shrinkage and selection operator (LASSO) regression, Cox regression, and nomogram. A nomogram represents a graphical tool employed within statistical learning to estimate the probability of an outcome or compute the value of a variable by considering the values of other associated variables [24]. Machine learning, a subfield of artificial intelligence (AI), is dedicated to creating algorithms and models that empower computers to learn from data and make predictions. Prominent machine learning techniques encompass support vector machine (SVM), decision tree, and random forest (RF). Deep learning, a subfield of machine learning, focuses on leveraging neural networks with multiple layers to comprehend intricate patterns and representations within data. The convolutional neural network (CNN) is a type of artificial neural network used in deep learning.
Radiomics is the extraction and analysis of numerous quantitative features from medical images, such as those obtained from computed tomography (CT), magnetic resonance imaging (MRI), and positron emission tomography with computed tomography (PET-CT), into computationally exploitable information [25,26]. This method entails extracting textural, shape-related, intensity-based, and spatial attributes from images. By employing machine learning or statistical learning techniques on the derived radiomic features, prediction of clinical outcomes becomes possible.
Study characteristics and predictive models
The PRISMA flow diagram (Fig. 2) summarizes the literature screening process. After removing duplicates, the search strategy yielded 117 studies for full-text screening. Seven articles [15,27-32] were included in this systematic review. Table 2 summarizes the baseline characteristics of the included studies. All seven articles were retrospective in design, published between 2019 and 2022, and comprised six single-center studies and one multi-center study with a total of 35,989 participants. Notably, two studies had sample sizes larger than 2,000 patients. The distribution of included predictive models was as follows: radiomics (n = 3), LR (n = 3), Cox regression (n = 2), nomogram (n = 3), SVM (n = 2), RF (n = 2), and CNN (n = 2).
Predictors and outcomes
The predictive outcome of all included studies was the occurrence of CRCLM. Three studies integrated radiomics with LR, SVM, or RF; three employed nomograms in conjunction with LR or Cox regression; and two employed CNN (Table 2). The incorporation of clinicopathological variables as significant predictors was a consistent practice observed across all included studies. Amongst the clinicopathological variables used for predicting CRCLM, age (n = 5), sex (n = 3), carcinoembryonic antigen (CEA, n = 3), N stage (n = 4), and T stage (n = 3) emerged as the most frequently used predictors. Additional clinical predictors reported by individual studies included tumor size, tumor location, chemotherapy, CT images, digital pathological images, vascular emboli, lymphatic invasion, perineurial invasion, family history of CRCLM, and the presence of Kirsten rat sarcoma viral oncogene homologue (KRAS) mutations.
Model performance
The discriminative performance of each predictive model was assessed using the AUC (Table 2). The AUC values, ranging from 0 to 1, were utilized to gauge the predictive performance of the models, with 0.5 denoting random chance, and 1.0 indicating a perfect fit [33]. AUC values surpassing 0.7 indicated a reasonably accurate prediction model [33]. Regarding the prediction of CRCLM, the mean AUCs ranged between 0.697 to 0.870, with 86% of models demonstrating clear discriminative ability (AUC > 0.70). Amongst the diverse models employed, the hybrid approach incorporating both clinical and radiomic features alongside SVM demonstrated the highest discriminative ability (AUC = 0.870). Other well-performing models included radiomics with SVM (AUC = 0.850), clinical features with RF (AUC = 0.860), the combined model integrating both clinical and radiomic features with RF (AUC = 0.860), as well as CNN model coupled with Cox regression and nomogram (AUC = 0.848).
Model evaluation
All included studies conducted internal validation of their models through the resampling methods (Table 2), specifically cross-validation (n = 5) and/or bootstrapping (n = 4). Resampling techniques play a crucial role in evaluating and validating predictive models, ensuring their performance reliability [34]. Cross-validation partitions the available data into distinct subsets, or folds, where the model is trained on one fold (training set), and evaluated on the remaining fold (validation set) [35]. This technique mitigates overfitting risks and gauges the model’s generalizability across diverse data subsets. Conversely, bootstrapping involves generating multiple bootstrap samples through random sampling with replacement from the original dataset [35]. These bootstrap samples, of the same size as the original dataset, enable the training and evaluation of multiple models. Bootstrapping’s diverse resampled datasets facilitate the estimation of model performance variability and calculation of confidence intervals for key metrics, such as accuracy or AUC. The calibration, which pertains to the correspondence between the anticipated probabilities derived from a predictive model and the actual observed outcomes, signifying the precision and dependability of the model’s predictions, was reported for two nomogram models. None of the included studies reported external validation.
Risk of bias
Fig. 3 and Supplementary Table 1, 2 present the results of RoB and applicability assessment. The overall RoB was determined to be high in 71% of the included studies. Within the participants domain, 57% of the studies were assessed as having a low RoB. However, one study [28] was identified as having a high RoB due to insufficiently detailed inclusion criteria, while two studies [30,31] were classified as having an unclear RoB for similar reasons. Regarding the domain of predictors, 43% of the studies were assessed as low RoB. Notably, four studies [15,27,29,30] that utilized radiomic features were assigned a high RoB, due to the inherent complexity associated with these features. As for the outcomes domain, 71% of the studies were assessed as low RoB, with two studies [15,29] identified as having a high RoB due to their relatively small sample sizes (< 100). For the domain of analysis, 71% of the studies were assessed as low RoB, while only one study [29] was deemed to have a high RoB due to a lack of information regarding the 95% confidence interval of AUC, and the absence of details concerning the predictor selection process, coupled with a small sample size.
DISCUSSION
This systematic review encompassed a comprehensive analysis of seven studies, collectively reporting 14 predictive models incorporating diverse risk factors for CRCLM. Our study sought to identify and evaluate the predictive models that exhibited promising discrimination capabilities for CRCLM. By considering the characteristics of the included studies, essential insights regarding the current research landscape of predictive modeling for CRCLM were obtained.
The retrospective design of all seven studies underscored the reliance on historical data to develop and validate predictive models. This approach allowed clinicians and researchers to leverage existing patient information to construct and assess the performance of the models [36]. Furthermore, all studies were published between 2019 and 2022, reflecting the recent interest in predictive modeling for CRCLM. The temporal proximity of these articles suggests an evolving and dynamic research landscape characterized by the ongoing pursuit of innovative strategies for predicting CRCLM.
The distribution of predictive models also exhibited considerable diversity with various statistical/machine/deep learning approaches, including LR, Cox regression, nomogram, SVM, RF, and CNN. The 14 predictive models demonstrated mean AUC values ranging from 0.697 to 0.870, with the majority (71%) achieving an AUC exceeding 0.75, indicating valuable discriminatory performance [37]. Three studies used radiomic features in conjunction with machine learning algorithms to predict CRCLM, with one study resulting in the highest AUC of 0.870 by combining clinical and radiomic features with SVM. This highlights the effectiveness of integrating multiple data sources and advanced computational techniques at enhancing the accuracy of CRCLM prediction.
CT, MRI, and PET-CT are frequently used imaging modalities to detect CRCLM. Nevertheless, their diagnostic sensitivity and accuracy can vary depending on equipment and reporting radiologist’s expertise [38]. In a meta-analysis with a 20-year study period, the detection sensitivities for CRCLM for CT, MRI, and PET-CT were reported as 74.4%, 80.3%, and 81.4%, respectively [39]. Radiomics has shown promise in surpassing the limitations of conventional imaging, by enabling quantitative and comprehensive analysis of tumor characteristics [40]. Its ability to capture hidden patterns and heterogeneity within the tumor microenvironment provides valuable insights into the risk and potential for CRCLM development. Our results showed that radiomic models employing quantitative image features extracted from CT or MRI achieved an AUC greater than 0.70. These findings are consistent with studies that used radiomic models to predict distant metastases in other types of cancer [41-43], suggesting that the combination of radiomic features and machine learning techniques has the potential to improve the predictive accuracy of CRCLM.
However, the application of radiomics in predicting CRCLM faces several challenges. One significant drawback is the potential variability in image features across healthcare settings. Variations in imaging protocols, equipment, and image acquisition techniques can lead to inconsistencies in the extracted radiomic features. As a result, a model developed and validated in one healthcare setting may not perform optimally when applied to another setting. To overcome the lack of generalizability, the integration of clinicopathological alongside radiomic characteristics becomes crucial.
Clinical attributes provide important contextual information that complements the image-based features captured by radiomics. While radiomic features may be influenced by site-specific variations, clinical attributes tend to have a more consistent definition. Age, sex, CEA levels, and tumor stage (T and N staging) were identified as the most widely used predictors for CRCLM. Age is a fundamental demographic feature that may serve as a proxy for multiple factors associated with disease pathogenesis and progression. Age-related alterations in the immune response and decreased immune surveillance, including impaired T-cell proliferation, increased CD8+ cytotoxic cell numbers, and decreased CD4+ T-cell and CD19+ B-cell numbers, have been postulated to affect the immune capacity to identify and eliminate metastatic CRC cells, thereby potentially contributing to an increased risk of CRCLM [44-49].
Similarly, sex disparities in metastatic CRC outcomes have been observed [50,51], establishing sex as a noteworthy predictor for CRCLM. Hormones, including estrogen and testosterone, have been implicated in CRC development and progression [52,53]. Sex-specific hormonal differences influence the susceptibility to CRCLM, as estrogen potentially exerts protective effects against CRCLM development [52,54]. CEA is associated with other key factors, such as large tumor size, advanced tumor stage, lymph node involvement, and its involvement in facilitating tumor cell adhesion, migration, and invasion processes [55-59]. The regular monitoring of CEA levels plays a pivotal role in identifying individuals at heightened risk of CRCLM, and aids in the formulation of appropriate surveillance and treatment strategies [60]. The depth of tumor invasion and lymph node involvement are also high-risk features and predictors for CRCLM development [61].
SVM, RF, and Cox regression with nomogram showed better performance than LR and CNN in the prediction of CRCLM. SVM can effectively handle complex and high-dimensional clinical and radiomic features, capturing intricate patterns and non-linear relationships, which is crucial for accurate predictions [62]. The ability of SVM to separate data points into different classes by finding an optimal hyperplane maximally distant from the data points of different classes enhances its predictive accuracy [63]. RF is an ensemble learning method that combines multiple decision trees to make predictions. By constructing a multitude of decision trees on random subsets of the data and aggregating their predictions, RF can mitigate overfitting, and improve the generalizability of the predictive model [64]. Cox regression with nomogram is a survival analysis technique that considers time-to-event data and covariates. By incorporating clinical variables and constructing a nomogram, which visually represents the contribution of each predictor, this approach allows estimation of individualized probabilities for developing CRCLM.
On the other hand, LR and CNN may exhibit comparatively lower prediction performance. LR assumes linear relationships between predictors and the outcome, which may not adequately capture the complex interactions and non-linear associations present. CNN, although powerful in image analysis tasks, may not fully exploit the relevant features for CRCLM prediction, as it primarily focuses on extracting spatial patterns from medical images instead of incorporating clinical variables. Incorporating CNN for digital pathological image analysis, followed by Cox regression and nomogram, can enhance the predictive accuracy of the model.
Despite the generally high accuracies observed in the predictive models assessed in this review, there remain several limitations. One notable shortcoming is the absence of external validation in all seven included studies. In addition, calibration, which provides information about the agreement between predicted probabilities and observed outcomes, was only reported in three studies (43%) that utilized nomograms. Poor calibration suggests potential under- or overestimation of the desired outcome by the model. Furthermore, the assessment of model performance primarily relied on discrimination measures, specifically AUC, as calibration measures were absent in four (57%) of the included studies. Due to the considerable heterogeneity amongst the included studies, conducting a pooled analysis or comparative meta-analysis of predictive models was not feasible.
Nonetheless, this review provides valuable insights into the current landscape of predictive models for CRCLM. Our findings highlight the potential of various algorithms by augmented by clinical and radiomic features in accurately predicting CRCLM. Future research could address the identified limitations by incorporating external validation in predictive model development. Efforts should also be made to improve the reporting of calibration measures, enhancing model performance and calibration accuracy. Furthermore, the heterogeneity of the included studies highlights the need for standardized methodologies and reporting guidelines in the field of predictive modeling. Developing consensus criteria and guidelines would facilitate more rigorous and comparable evaluations of predictive models, facilitating more robust evidence synthesis and meta-analyses. Collaborative efforts, particularly multi-center studies, are essential to enhance the generalizability and clinical utility of predictive models for CRCLM.
Conclusion
This review demonstrates the potential of predictive modeling for CRCLM. The integration of clinicopathological and radiomic features with machine learning algorithms showed superior predictive capabilities. External validation studies are necessary to establish the reliability and generalizability of predictive models, particularly across diverse healthcare settings. Improved reporting and standardized methodologies are also required to facilitate the integration of predictive models into routine clinical practice.
SUPPLEMENTARY DATA
Supplementary data related to this article can be found at https://doi.org/10.14701/ahbps.23-078.
XML Download