Between January 2001 and December 2016, 995 T1 CRC patients underwent endoscopic or surgical resection at the National Cancer Center, Korea. The exclusion criteria included preoperative chemoradiation therapy (n=3), synchronous advanced CRC (n=85), metastasis (n=8), and loss of follow-up (n=5). Patients with T1 CRC who had undergone endoscopic resection and a follow-up period of < 24 months, were also excluded (n=55). The patients whose the status of LNM could not be estimated were excluded as well (n=6). Finally, a total of 833 T1 CRC patients (154 endoscopic resections and 679 surgical resections) were included in the development set. The database of the National Cancer Center and patients’ clinical charts were reviewed retrospectively (Fig. 1).

For external validation, we utilized the CRC database from Daehang Hospital, a specialist colorectal hospital. Based on the same inclusion criteria as that of the development set, a total of 722 T1 CRC patients (249 endoscopic resections and 473 surgical resections) were included in the validation set. The database of the Daehang Hospital and patients’ clinical charts were reviewed retrospectively (Fig. 2).

Endoscopic tumors were classified into two types: polypoid and non-polypoid, according to the Paris classification [15]. The polypoid type include pedunculated (0-Ip) and sessile (0-Is) tumors. The non-polypoid type include slightly elevated (0-IIa), flat (0-IIb), slightly depressed (0-IIc), and excavated (0-III) tumors. The tumor locations were classified into three groups, namely, right colon (cecum to splenic flexure), left colon (splenic flexure to the rectosigmoid junction), and rectum (rectosigmoid junction to the anal verge). In surgically resected specimens, the depth of submucosal invasion (SM depth) was evaluated using Kudo’s classification, as follows: infiltration into the upper third (sm1), middle third (sm2), or lower third (sm3) of the submucosal layer. For endoscopically resected sessile and flat tumors, the cut-off between sm1 and sm2 was 1,000 μm, according to the Paris classification, with an SM depth > 2,000 μm defined as sm3. For endoscopically resected pedunculated tumors, the cut-off between sm1 and sm2 was at the level of the neck, and an SM depth > 3,000 μm from the neck was defined as sm3. Deep submucosal invasion was defined as an SM depth ≥ sm2. Differentiation of adenocarcinomas was classified according to World Health Organization criteria: grade 1 (well differentiated), grade 2 (moderately differentiated), or grade 3 (poorly differentiated) [16]. Grades 1 and 2 were defined as histologically low grade tumors, while a grade 3 tumor, a mucinous carcinoma, signet ring cell carcinoma, and carcinoma with neuroendocrine differentiation were defined as histologically high-grade tumors. An isolated cell or a small cluster of < 5 tumor cells in the invasive front was defined as a “budding” focus, and > 10 budding foci viewed at 200× magnification was defined as budding positive [17]. Vascular invasion was defined as the presence of cancer cells within endothelial-lined channels, including angiolymphatic invasion and venous invasion. Vascular invasion of small vessels without a vascular smooth muscle layer was defined as angiolymphatic invasion, and vascular invasion of large vessels with a vascular smooth muscle layer was defined as venous invasion. Background adenoma (BGA) was defined as an adenomatous component microscopically contiguous to the resected carcinoma. Clinicohistological parameter details were evaluated with respect to the LNM status.

The LNM status of the endoscopically resected group was determined indirectly, based on follow-ups and pathological results of salvage surgery.

Patients in the endoscopically resected group were followed-up by colonoscopy, performed between 3 and 6 months after resection and annually thereafter, and with an annual computed tomography scan of the abdomen and chest and an annual measurement of serum carcinoembryonic antigen concentration. Patients were followed-up for a minimum of 24 months.

Patients with no evidence of recurrence during the follow-up period were regarded as negative for LNM. In patients with recurrence, those diagnosed as LNM negative (rpN0) on salvage surgery were regarded as initially LNM negative, while those diagnosed as LNM positive (rpN+) on salvage surgery were excluded because it was not possible to demonstrate that LNM was already existed at the time of the initial endoscopic procedure or that LNM had developed in the regional area following the endoscopic procedure. The patients with recurrence who did not undergo salvage surgery due to multiple metastases were also excluded due to the possibility of skip metastases.

The distributions of risk factors affecting LNM were summarized as mean±standard deviation, or median (interquartile range [IQR]) for continuous variables, and frequency (percentages) for categorical variables. The differences of characteristics between development and validation datasets were tested using an independent t test or a Wilcoxon rank-sum test for continuous variables and Pearson’s chi-squared test for categorical variables. A logistic regression model was used to construct a model to predict the LNM. First, we examined whether 10 risk factors affected LNM in the univariable model, and then all variables were included in the multivariable model. The final multivariable prediction model was determined using a backward variable selection method with an elimination criterion of a p > 0.05. To assess the performance of the prediction model, discrimination was evaluated using the receiver operating characteristic (ROC) curves with area under the ROC curve (AUC), and calibration was assessed using the Hosmer-Lemeshow (HL) goodness-of-fit test. The prediction model is considered good in discrimination when the AUC is greater than 0.75. Model calibration assessment was considered adequate when p > 0.05. The validation of the prediction model was conducted using internal and external validation. First, the internal validation was performed using a bootstrap approach of 2,000 times resampling to correct the bias [18]. The external validation was performed using an independent new dataset to support generalizability. We illustrated the nomogram to facilitate the interpretation of the results, and calculated the predicted probability of LNM risk for individuals. p-values < 0.05 were considered statistically significant. All statistical analyses were performed using SAS ver. 9.4 (SAS Institute Inc., Cary, NC) and R software, ver. 3.3.3 (R Project for Statistical Computing, Vienna, Austria).

This study was approved by the Institutional Review Board of the National Cancer Center, Korea (NCC 2017-0189) and Daehang Hospital (DH17-0002). Informed consent was obtained from all patients.

Among the 833 patients included in development database set, 160 underwent endoscopic resection and 679 underwent surgical resection. Among the 160 endoscopically resected cases, 150 had no recurrence, and were regarded as LNM negative. Among 10 patients who had recurrence after endoscopic resection, four were diagnosed as LNM negative (rpN0) on salvage surgery, and considered initially LNM negative. Five patients diagnosed as LNM positive on salvage surgeries, and one who did not undergo salvage surgery due to multiple metastases, were excluded (Fig. 1).

Among the 722 patients included in validation database set, 249 underwent endoscopic resection and 473 underwent surgical resection. Among the 249 endoscopically resected cases, all 249 had no recurrence, and were regarded as LNM negative (Fig. 2).

The incidence rate of LNM was 11.6% (97/833) in the development set and 6.8% (49/722) in the validation set. The mean age was significantly lower in the development set than in the validation set (61.8±10.6 vs. 67.8±10.6, p < 0.001). The proportion of endoscopic-type tumors was also different between the two datasets (p=0.005). For treatment, more patients underwent additional surgery after endoscopic resection than for endoscopic resection only or for surgery only. The median and IQR (Q1-Q3) of pathologic tumor size was 14 mm (10-20 mm) in the development set and 19 mm (14-27 mm) in the validation set (p < 0.001). The SM depth was distributed evenly in the development set, but the percentage of sm1 was greater (44.0%) in the validation set. There was a statistically significant difference in the incidence of vascular invasion between the development and validation sets (44.1% vs. 16.3%, p < 0.001). No differences of distribution were observed regarding sex, tumor location, histologic grade, background adenoma, and tumor budding between the development and the validation sets (Table 1).

To construct the LNM risk prediction model, 10 candidate predictors were examined in univariable and multivariable logistic regression models (Table 2). As a result, 5 risk factors were included in the final multivariable model. Vascular invasion and high-grade histology were the strongest risk factors (odds ratio [OR], 8.45; 95% confidence interval [CI], 4.56 to 15.66); p < 0.001 and OR, 7.89; 95% CI, 2.89 to 21.52; p < 0.001, respectively). Deep submucosal invasion (sm2/3) and tumor budding were also statistically significant predictors of LNM (OR, 2.14; 95% CI, 1.19 to 3.86; p=0.012 and OR, 1.70; 1.03 to 2.80; p=0.039, respectively). The absence of BGA was associated with 42% lower odds of LNM (OR, 0.58; 95% CI, 0.36 to 0.92; p=0.022).

The prediction model using the development set was evaluated with good performance in terms of discrimination and calibration (Fig. 3). The AUC was 0.812 (95% CI, 0.770 to 0.855) and the hazard ratio chi-square test statistic was 1.266 (p=0.737). The bias corrected AUC was 0.794 in internal validation using the bootstrap method. Due to the limitations of internal validation to determine the generalizability of a prediction model, we conducted external validation using an independent new dataset. As a result, the AUC dropped to 0.771 (95% CI, 0.708 to 0.834) and the p-value of the HL chi-squared test was 0.040. Although the validation dataset was very different from the development dataset, the performance of the model remained good.

To facilitate user-friendly graphical interfaces, we constructed the nomogram with the final prediction model (Fig. 4). Through ranking the effect estimators, point scores were assigned to each risk factor. The total points accumulated from all the risk factors corresponded to the predicted probability of LNM for individual patients. The incidence rate of LNM was 11.6% in this study; therefore, we suggested sensitivity, specificity, a PPV, and a negative predictive value at each cut-off point to be between 5% and 15% in units of 1% (S1 Table). We calculated the probability of LNM occurrence for all combinations generated using the five risk factors of the final prediction model (S2 Table).